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Getting  to  Local  Information:  The  Roles  of  Different 

Costs 

1.  Introduction 


Computer  users  who  spend  considerable  time  processing  infonnation  (termed 
knowledge  workers  in  this  thesis)  access  much  of  this  information  on  computer 
systems.  Unfortunately,  the  task  of  accessing  such  resources,  even  when  stored 
locally  in  files  and  folders  on  the  user’s  computer,  is  often  not  a  simple  one.  One 
reason  may  be  that  the  information  may  not  be  organized  according  to  their  current 
needs  —  or  not  even  be  very  well  organized  by  any  criterion.  Several  research  efforts 
have  helped  to  shed  light  on  user  processes  for  organizing  and  retrieving  information 
[Bannon  et  al  1983,  Malone  1983,  Boardman  and  Sasse  2004],  and  the  difficulties 
they  run  into  along  the  way  [Barreau  et  al  1995,  Bruce  et  al.  2004,  Jones  et  al.  2005, 
Kaptelinin  2003,  Ravasio  et  al.  2004]. 


The  good  news  is  that  the  knowledge  worker  has  a  number  of  choices  about  how  to 
access  a  resource  on  her  computer,  and  at  least  one  of  them  is  likely  to  be  very 
efficient.  Unfortunately,  this  is  also  the  bad  news.  For  example,  she  might  get  to  it 
through  the  "Recent  Documents"  list,  or  by  navigating  from  an  open  Windows 
Explorer  window.  Each  choice  incurs  different  costs  and  risks  for  her  —  navigating  is 
straightforward  but  can  be  time-consuming,  looking  through  the  "Recent  Document" 
list  takes  time  and  the  resource  might  not  be  listed  in  it,  and  so  on.  Clearly,  all  of 
these  choices  carry  different  costs  and  risks  of  failure,  and  considering  each  in  order 
to  choose  the  best  seems  warranted,  yet  adds  a  cost  of  its  own. 


Recent  improvements  in  desktop  search  do  not  eliminate  the  problem:  users  often  still 
navigate  around  manually.  For  example,  previous  research  has  found  that  users  prefer 
manual  navigation  to  search  in  many  cases,  especially  when  it  is  difficult  to  produce 
appropriate  search  terms  such  as  file  names;  that  novices  particularly  prefer  manual 
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navigation  over  search;  and  that  even  skilled  users  use  search  only  occasionally 
[Barreau  et  al  1995,  Ravasio  et  al.  2004]. 


In  response  to  the  reported  difficulties  users  have  in  accessing  their  resources,  there 
have  been  several  research  efforts  to  help  users  better  organize  and  retrieve 
information  [Bellotti  et  al.  2003,  Czerwinski  et  al.  2004,  Dragunov  et  al.  2005, 
Kaptelinin  2003,  Smith  et  al.  2003].  This  has  led  to  a  proliferation  of  approaches  to 
support  access  to  resources.  However,  these  works  have  not  attempted  to  distinguish 
the  types  of  costs  of  users’  difficulties,  illuminate  how  different  types  of  costs  impact 
users’  behaviors,  or  pin  down  precisely  which  of  the  costs  could  be  reduced  by  the 
proposed  solutions. 


We  would  like  to  shed  some  light  on  these  issues,  so  that  technologies  aiming  to 
reduce  user  costs  have  a  clearer  target  at  which  to  aim.  We  conceptually  define  cost 
to  mean  the  user’s  time  and  effort  spent  on  getting  to  local  information.  We  consider 
three  different  types  of  cost:  (1)  actual  costs  borne  by  the  users,  (2)  optimal  costs,  i.e., 
the  minimum  amount  of  time  and  effort  required  to  access  a  resource  under  specific 
assumptions,  and  (3)  perceived  costs,  i.e.,  the  users’  opinions  of  the  costs  of  different 
ways  to  access  resources.  (Operational  definitions  of  these  terms  will  follow.)  We  use 
the  Attention  Investment  Model  [Blackwell  2002]  as  a  basis  to  understand  the  cost- 
risk-benefit  decisions  made  by  users  while  accessing  resources. 


In  this  thesis,  we  first  present  a  cost-centric  methodology  for  investigating  these  costs. 
We  then  use  this  methodology  and  present  empirical  results  of  an  experiment  that 
specifically  investigated  the  roles  of  the  three  different  types  of  costs  on  users’ 
choices.  Our  results  demonstrate  that  careful  consideration  of  the  roles  of  these  three 
different  types  of  costs  can  reveal  useful  insights  that  help  inform  technologies  in  the 
domain  of  interfaces  for  local  resource  access. 
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2.  Background 
2.1  TaskTracer 

TaskTracer  is  a  desktop  software  system  that  tracks  in  detail  how  knowledge 
workers  work  on  tasks  and  uses  this  infonnation  intelligently  to  increase  their 
productivity  and  efficiency.  The  main  goals  of  TaskTracer  are  to  make  user  interfaces 
task-aware,  help  with  interruption  recovery  and  improve  personal  infonnation 
management. 


The  central  concept  of  TaskTracer  is  that  knowledge  workers  organize  their  work 
into  discreet  work  units  called  tasks.  This  system  supports  Task  Management  by 
grouping  information  automatically  into  tasks.  Knowledge  workers  continually 
consume  and  produce  information.  This  information  is  usually  available  in  different 
locations  and  has  to  be  accessed  using  different  applications  such  as  Microsoft  Word, 
Microsoft  Excel,  Microsoft  PowerPoint,  Web  browsers  etc.  TaskTracer  helps  to 
bring  all  the  required  information  from  different  locations  and  different  application 
types  to  one  place  by  associating  all  this  information  with  a  task.  Thus  one  of  the 
main  goals  of  TaskTracer  is  to  help  the  users  to  easily  access  resources  associated 
with  a  task. 


The  resources  associated  with  a  task  are  displayed  in  a  list  in  the  “TaskExplorer” 
which  is  the  UI  component  of  TaskTracer  (See  Figure  1).  As  we  can  see  from  the  UI 
(Figure  1),  all  the  tasks  are  displayed  in  the  left  pane.  On  selecting  a  particular  task, 
all  the  resources  associated  with  that  task  can  be  viewed  under  the  “All”  tab.  To  view 
resources  of  a  particular  application  type,  we  can  choose  the  corresponding  tab. 


Knowledge  workers  multi-task.  As  they  are  working  on  their  tasks  they  are 
continuously  trying  to  locate  and  open  resources  associated  with  that  task.  Such 
actions  of  locating  and  opening  resources  itself  might  lead  to  an  overhead  cost.  In 


4 


B  Task  Explorer 


Task  View  Tools  Help 


JSJxJ 


B-ffi 

ExplanationsStudy 
I  ^  HCI 
:  IntelStudy 

Interviews 
1  -j5f  Personal 
:  iyj  Test 
B  Thesis 

^  Related  Work  and 
ivj  UlSTPaper 
• . Unknown 


±1 


All 


Web 


Folders  Word  Excel  PowerPoint  Mail 


Contacts  Other 


Mail ::  Message  Composition 
. . .  webmail.  oregonstate.  edu /. . . 
Mail ::  Message  Composition 
Mail ::  Message  Composition 
Z: V . .  onskAnaly$is\M  arkersE . . . 
Subjectl  0_Carey_Vidya.xls 
Z:  V . .  dya\T  T  \E  xplanations\A. . 
Z: V . .  anations\E  xperimentM  a. . 
Z:  Widya  WidyaM  RBMR  B_ma. . . 
Subjectl  1_Allison_Vidya.xls 
S  ubjectl  3_Agreement_R  uss. . 

. . .  webmail.  oregonstate.  edu/. . . 
0utline_0622_1 1 1 5.doc 
Methodology  for  Deriving  Co.. 
0utline_0622.doc 
. . .  webmail.  oregonstate.  edu/. . . 
Mail ::  Message  Composition 
. . .  webmail.  oregonstate.  edu/. . . 

. . .  webmail.  oregonstate.  edu/. . . 
Mail ::  Message  Composition 
Codes_0608.doc 
Z:  V . .  aWidya\T  T  \E  xplanatio. . . 
C:\-..v\Application  Data\SS... 
Z:  Widya  Widya\T  T  \E  xplanati. . 
Z:  \. . .  telS  tudy\E  xperimentM  at. . 
Logout 

Look-Up  Classes  to  Add: 
Look-up  Classes  to  Add  JWF 
Add/Drop  Classes: 

Select  Term 
Registration 


|  Path 

Type 

|  Date  Modified 

. . .  webmail.  oregonstate.  edu/compose.  php?u. . . 

1  Explorer 

6/23/200B  2... 

. . .  webmail.  oregonstate.  edu/compose.  php?u. . . 

1  Explorer 

B/23/200G  2... 

. . .  webmail.  oregonstate.  edu/compose.  php?u. . . 

1  Explorer 

6/23/2006  2... 

. . .  webmail.  oregonstate.  edu/compose.  php?p. . . 

1  Explorer 

6/23/200B  2... 

.  Z:  V . .  E  xplanations\Analysis\M  arkersE  xported\ 

Folder 

G/23/200G  2... 

Z: V . .  E  xplanations\Analy$is\M  arkersE  xported\ 

Excel 

B/23/200G  2... 

. .  Z:  Widya  Widya\T  T  \E  xplanationsSAnalysis\ 

Folder 

6/23/2006  2... 

.  Z  . .  a\T  T  \E  xplanations\E  xperimentM  aterials\ 

Folder 

6/23/2006  2... 

.  Z:  Widya  Widya\IRB\IRB_main\ 

Folder 

6/23/2006  2... 

Z: V . .  E  xplanations\Analysis\M  arkersE  xported\ 

Excel 

6/23/2006  2... 

.  Z:  V . .  E  xplanations\Analysis\M  arkersE  xported\ 

Excel 

6/22/20061... 

. . .  webmail.  oregonstate.  edu/compose.  php?u. . . 

1  Explorer 

6/22/20061... 

Z . .  aWidya\T  T  \E  xplanation$\CALO  Write. . . 

Word 

6/22/20061... 

.  Z:  V . .  aWidya\T  T  \E  xplanations\CALO  Write. . . 

Word 

6/22/20061... 

Z: V . .  a  Widya\T  T  \E  xplanations\CALO  Write. . . 

Word 

6/22/20061... 

. . .  webmail.  oregonstate.  edu/ compose.  php?u. . . 

1  Explorer 

6/22/20061... 

. . .  webmail.  oregonstate.  edu/compose.  php?p. . . 

1  Explorer 

6/22/20061... 

. . .  webmail.  oregonstate.  edu/compose.  php?u. . . 

1  Explorer 

6/22/20061... 

. . .  webmail.  oregonstate.  edu/compose.  php?u. . . 

1  Explorer 

6/22/20061... 

. . .  webmail.  oregonstate.  edu/compose.  php?p. . . 

1  Explorer 

6/22/20061... 

C:V..amv.EECS\ApplicationD  ata\S  S  H  \temp\ 

Word 

6/22/20061... 

Z:  Widya  Widya\T  T  \E  xplanations\D  ata\ 

Folder 

6/22/20061... 

C:V..$\rajaramv\ApplicationData\SSH\temp\ 

Folder 

6/22/20061... 

. .  Z:  Widya Widya\T  T  \E  xplanations\ 

Folder 

6/22/20061... 

. .  Z  \. . .  dya\T  T  \l  ntelS  tudy\E  xperimentM  aterials\ 

Folder 

6/22/20061... 

. . .  adminf  o.  ucsadm.  oregonstate.  edu/prod/tw. . . 

1  Explorer 

6/22/2006  9... 

. . .  adminf  o.  ucsadm.  oregonstate.  edu/prod/b. . . 

1  Explorer 

6/22/2006  9... 

. . .  adminf  o.  ucsadm.  oregonstate.  edu/prod/b. . . 

1  Explorer 

6/22/2006  9... 

. . .  adminf  o.  ucsadm.  oregonstate.  edu/prod/b. . . 

1  Explorer 

6/22/2006  9... 

. . .  adminf  o.  ucsadm.  oregonstate.  edu/prod/b. . . 

1  Explorer 

6/22/2006  9... 

. . .  adminf  o.  ucsadm.  oregonstate.  edu/prod/tw. . . 

1  Explorer 

6/22/2006  9... 

Figure  1:  TaskExplorer  -  The  resources  associated  with  the  “Explanations- 

Study”  task  is  displayed  as  a  list 

order  to  avoid  such  overhead  costs  and  to  help  with  personal  information 
management,  the  TaskTracer  UI  has  been  designed  to  minimize  the  users’  cognitive 
and  physical  costs  of  locating  information.  In  order  to  inform  the  design  of 
TaskTracer,  this  thesis  is  focused  towards  identifying  the  users’  current  strategies  and 
nature  of  overhead  costs  associated  with  accessing  resources  in  the  traditional 
windows  environment. 


2.2  User  Costs 

2.2.1  Attention  Investment  Model 

Our  investigation  makes  use  of  the  Attention  Investment  Model  [Blackwell 
2002]  as  a  basis  for  understanding  costs.  Attention  Investment  Model  is  a  cognitive 
model  that  analyzes  people’s  decisions  about  how  to  perfonn  tasks  based  on  their 
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perceptions  of  costs,  benefits  and  risks.  This  predictive  model  describes  how  users 
decide  to  allocate  their  attention  by  conducting  a  cost-risk-benefit  analysis  during 
problem-solving,  in  which  the  units  they  are  weighing  up  are  the  cost  of  their 
attention.  In  this  model,  risk  is  expressed  as  a  probability  of  wasted  attention  or 
unanticipated  additional  attention.  Cost  and  benefits  are  expressed  in  "attention 
units",  a  concept  similar  to  time  that  also  includes  the  notion  of  degree  of  on-task 
focus  by  the  user.  In  these  units,  the  expected  benefit  of  an  action  is  compared  to  the 
investment  required  and  to  the  risks  that  the  cost  may  be  paid  without  receiving  the 
expected  benefits  after  all. 


Variable 

Attention  Investment  Model 

Examples  from  Our  Work 

Cost 

Perceived  attention  units  to  get  the 

work  done 

Perceived  cost  of  actually 

finding  the  file  by  going 

through  that  path 

Investment 

Perceived  attention  units  expended 

towards  a  potential  reward 

•  Perceived  cost  of 

choosing  a  path 

•  Perceived  cost  of 

learning  to  use  a 

particular  path 

Pay-off 

Perceived  reduction  in  future  cost 

that  will  result  from  the  way  the 

user  has  chosen  to  spend  attention 

Perceived  time  savings  of 

finding  the  file  by  going  down 

the  path  chosen 

Risk 

Perceived  probability  that  no  pay¬ 
off  will  result  or  even  that 

additional  future  costs  will  be 

incurred 

Perceived  probability  of  not 

finding  the  file  by  going  down 

a  particular  path  and  of  extra 

time  being  needed  to  backtrack 

and  go  down  a  new  path. 

Table  1:  AIM  Variables 


6 


The  variables  involved  in  the  cost-benefit  analysis  of  the  Attention  Investment 
Model  and  their  application  to  our  work  are  shown  in  Table  1. 

The  Attention  Investment  Model  focuses  on  perceived  costs,  risks,  and  benefits. 
This  aspect  of  the  model  is  critical:  often  users  make  choices  based  on  limited 
information.  The  users’  perception  of  costs,  risks,  and  benefits  may  be  influenced  by 
their  past  experiences,  and  may  have  little  to  do  with  the  actual  costs,  risks,  and 
benefits.  The  perceived  amount  of  benefit  includes  not  only  the  immediate  benefit, 
but  also  the  extent  to  which  the  user  believes  that  their  investment  of  time  may  reduce 
future  costs. 

The  Attention  Investment  Model  will  serve  as  an  intellectual  framework  in  our 
investigation  into  how  people  make  choices  among  ways  to  access  resources.  In 
particular,  we  are  interested  in  understanding  how  peoples’  decisions  and  the 
strategies  they  use  in  accessing  resources  relate  to  perceived  cost,  risks  and  benefits 
as  pointed  out  by  the  Attention  Investment  model.  Additionally  we  are  interested  in 
studying  the  relationship  between  their  perceived  costs,  their  actual  costs  and  the 
optimal  costs. 

2.2.2  Attentional  User  Interface 

Closely  related  to  the  Attention  Investment  Model  is  the  Attentional  User 
Interface  (AUI)  model  [Horvitz  et  al.  2003].  The  AUI  model  also  draws  from 
economic  models  of  attention  and  investment.  The  AUI  model  takes  into  account  the 
information  content  of  a  potential  interruption  by  the  computer  (such  as  a  potential 
notification  of  an  important  incoming  email  message)  in  the  context  of  the  user’s 
preferences  and  his/her  current  and  recent  online  activities  as  detected  by  sensors. 
Although  the  AUI  model  does  not  consider  cognitive  activities  directly,  the  user’s 
physical  activities  can  be  regarded  as  indirect  clues  about  the  user’s  attention.  The 
AUI  model’s  purpose  is  to  predict  whether  and  how  the  computer  should  interrupt  the 
user.  From  the  perspective  of  our  research,  this  is  a  critical  difference  between  the 
AUI  model  and  the  Attention  Investment  Model  —  the  Attention  Investment  Model 
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considers  what  the  user  will  do  under  specific  conditions,  whereas  the  AUI  model 
considers  what  the  computer  should  do. 


2.3  Finding  Files 

Our  work  builds  on  the  findings  of  previous  researchers  who  have  investigated  the 
preferences  and  habits  of  people  attempting  to  access  local  information,  and  how  well 
their  habits  align  with  the  mechanisms  available.  For  example,  numerous  studies  have 
found  that  people  rely  heavily  on  the  classification  of  information  into  folders  to 
facilitate  retrieval,  preferring  manual  browsing  over  logical  search  [Barreau  et  al. 
1995,  Jones  et  al.  2005,  Ravasio  et  al.  2004].  One  reason  for  this  may  be  that  users 
prefer  to  navigate  to  a  desired  file  in  small  steps  using  context  as  a  guide  [Teevan  et 
al.  2004].  Furthermore,  users  employ  a  limited  number  of  manual  browsing 
techniques  [Kaptelinin  1996]. 


Keeping  files  and  folders  maintained  in  logical  hierarchies  costs  users  time.  To 
save  this  time,  various  approaches  for  personal  information  management  that  do  not 
rely  on  hierarchically  structured  file  systems  have  been  proposed.  For  example,  one 
organization  scheme  is  the  "pile"  or  "data  mountain",  a  loosely  structured  information 
collection  [Dourish  et  al.  1999,  Robertson  et  al.  1998].  Another  approach  is  to  show 
information  in  a  time-centric  visualization  [Freeman  et  al.  1996,  Rekimoto  et  al. 
1999]. 


Time-centric  organizations  have  the  advantage  of  allowing  the  worker  to  return  to 
records  of  past  work,  but  for  multi-tasking  workers,  a  single  timeline  is  a  jumble  of 
multiple  tasks.  Because  projects,  activities,  and  to-do  items  play  an  important  role  to 
the  knowledge  worker  [Bannon  et  al  1983,  Gonzalez  et  al.  2004],  various  tools  have 
been  developed  to  support  grouping  information  in  task-centric  workspaces  [ 
Dragunov  et  al.  2005,  Kaptelinin  2003,  Smith  et  al.  2003,  Boardman  et  al.  2003, 
Henderson  and  Card  1986,Ringel  2003]. 


Further  information  on  the  specific  costs  that  users  incur  could  be  used  to  help 
these  efforts  to  minimize  costs,  and  helping  to  fill  this  gap  is  the  goal  of  this  thesis. 
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3.  Research  Approach 

To  investigate  how  people  access  local  resources  and  their  reasons  for  choosing 
different  ways  of  accessing  them,  we  developed  a  methodology  that  combines 
analytical  and  empirical  devices  and  that  distinguishes  actual,  optimal,  and  perceived 
costs. 


We  used  this  methodology  to  conduct  an  experiment  to  answer  the  following  research 
questions: 

•  Among  how  many  different  ways  must  users  choose  to  access  resources? 

•  What  roles  do  actual,  optimal,  and  perceived  costs  play  in  the  choices  that 
users  make? 

We  used  the  Attention  Investment  Model  as  an  intellectual  framework  for  the 
investigation,  considering  how  the  results  might  be  interpreted  using  this  model  and 
what  implications  ensue  for  information  access  technologies. 
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4.  A  Cost-Centric  Methodology  for  Investigating  Resource  Access 

The  methodology  we  present  here  consists  of  a  model  for  locating  resources  plus 
three  devices  that  can  measure  actual,  optimal  and  perceived  costs.  (We  enumerate 
complete  details  of  the  model  in  Appendix  D). 

4.1  The  Model:  Possible  Alternatives 

The  core  of  our  methodology  is  a  state-transition  model  that  reflects  all  the  states 
and  transitions  for  accessing  local  resources  for  a  standard  Windows  system.  The 
state-transition  model  reflects  possible  paths  a  user  can  take  through  this  state  space 
to  access  a  resource,  from  root  (deciding  what  first  step  to  use)  to  leaf  (finally 
opening  the  desired  file).  Each  transition  represents  a  choice  to  the  user  at  that  stage 
as  to  how  to  proceed  next. 


In  this  thesis,  we  use  an  abbreviation  convention,  in  which  each  grey  node  is  a 
supernode  —  a  collapsed  version  of  several  states  and  transitions.  Transitions  marked 
Decide  denote  mental  operations,  those  marked  Select  denote  physical  user  actions 
such  as  mouse  clicks  and  keypresses,  and  those  marked  Navigate  are  a  shorthand 
notation  indicating  that  both  a  mental  operation  to  decide  how  to  proceed  and  the 
sequence  of  physical  user  actions  to  reach  the  next  state  are  included. 


The  model  explicitly  depicts  the  work  required  for  the  user  to  access  a  local 
resource,  which  can  sometimes  be  quite  a  lot.  For  example,  consider  the  path  of 
opening  a  resource  using  My  Recent  Documents  from  the  Start  menu  (Figure  2). 
Thus,  in  this  example,  first  the  user  decides  how  to  proceed  from  the  top  level, 
transitioning  to  the  Start  Menu  Chosen  state.  The  user  then  takes  the  actions  to 
activate  the  Start  Menu  (Select  transition).  The  user  then  navigates  to  the  My  Recent 
Documents  supemode.  There  are  then  three  more  transitions  within  this  supernode 
required  to  open  the  resource.  At  any  time,  the  user  could,  of  course,  decide  to  try  a 
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Figure  2:  State  Diagram.  States  and  transitions  involved  in  opening  a  file  using  the 

Start  Menu’s  My  Recent  Documents. 

Supemode  (grey  node):-  A  collapsed  version  of  several  states  and  transitions 
Decide:  -  Mental  operations 
Select:  -  Physical  user  actions 

Navigate:  -  Includes  mental  operations  to  decide  how  to  proceed  and  the  sequence 
of  physical  user  actions  to  reach  the  next  state. 

different  approach.  This  is  depicted  in  the  figure  by  the  Decide  transition  leading  out 
of  the  largest  supemode  up  to  the  top  level. 


We  chose  to  combine  access  paths  with  only  trivial  differences  into  groups,  in 
order  to  make  analysis  tractable  and  useful.  Such  groups  of  access  paths  will  be 
referred  to  as  alternative  paths  in  this  paper.  Each  alternative  path  is  named  in  the 
form  of  Leaf  Node  (Root  Node).  For  example,  if  a  resource  was  opened  from  My 
Recent  Documents,  which  was  in  turn  reached  from  the  Start  Menu  as  per  Figure  2, 
this  alternative  path  would  be  named  My  Recent  Documents  (Start  Menu). 
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This  model  can  be  used  as  a  "stand-alone"  analytical  tool  for  cost-based 
investigation  into  resource  access.  For  example,  this  model  can  be  combined  with  a 
GOMS-based  [Card  et  al.  1980]  approach  to  provide  estimates  of  likely  completion 
times  for  each  alternative  path  by  summing  up  the  transitions  (operators,  in  GOMS 
tenninology  [See  Table  2]).  The  transition  counts  can  be  used  as  order  statistics  of  all 
the  existing  alternative  paths.  Given  such  a  model,  new  alternative  paths  can  be  added 
to  the  state-transition  model,  allowing  user  costs  of  the  new  alternatives  to  be 
estimated. 


Operator 

Time  required  to  perform  this 

operation  (seconds) 

K  (Keystroke) 

0.28 

P  (Point  with  mouse  to  something  on  the  display 

1.1 

B  (Press  or  release  mouse  button) 

0.1 

BB  (Click  a  mouse  button) 

0.2 

H  (Home  hands  either  to  the  keyboard  or  mouse) 

0.4 

M  (Mental,  thinking) 

1.35 

Table  2:  GOMS  -  taken  from  [Card  et  al.  1980] 

Consider  three  useful  subsets  of  alternative  paths.  All  possible  in  the  Windows 
environment  are  termed  possible  alternative  paths.  Not  all  possible  alternative  paths 
are  available  to  the  user  in  a  particular  context  (e.g.,  a  resource  may  not  be  in  the  My 
Recent  Documents  list);  this  subset  of  possible  alternative  paths  is  tenned  available 
alternative  paths.  The  choices  actually  made  by  users  are  their  chosen  alternative 
paths. 


The  model  answers  questions  about  only  one  of  these  subsets  in  a  way  that 
generalizes  to  all  users:  possible  alternative  paths.  In  order  to  also  investigate  actual 
and  optimal  costs  for  a  given  situation  requires  inclusion  of  the  other  two  subsets. 
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Thus  an  instantiation  of  the  model  is  needed,  taking  into  account  available  alternative 
paths  and  chosen  alternative  paths  based  on  a  user’s  concrete  context  and  actions. 
This  information  can  be  obtained  from  the  following  three  measuring  devices. 

4.2  Measuring  Actual  Costs 

The  first  device  measures  alternative  paths  chosen  by  users,  along  with  actual 
costs,  which  we  operationally  define  here  as  time  spent  to  open  a  resource. 
TaskTracer  [Dragunov  et  al.  2005]  was  used  to  record  each  participant’s  actions  in 
our  experiment,  but  similar  systems  can  achieve  the  same  result.  The  key  is  to  record 
each  action  and  the  time  needed  to  complete  those  actions,  generating  an  electronic 
transcript  of  each  participant’s  session.  TaskTracer  listened  in  on  all  user  interaction 
events  such  as  keypresses,  mouse  clicks,  and  window  focus  changes.  File  open  and 
close  events  were  also  recorded.  In  our  instantiation  of  the  device,  actual  costs  for 
individual  resource  accesses  were  measured  in  seconds  elapsed  from  the  time  of  the 
first  interaction  event  to  look  for  a  resource  in  the  electronic  transcript  (state  in  the 
model)  to  the  time  of  the  file  open  event. 


This  device  allows  the  investigation  of  "in  usage"  cost  by  providing  a  way  to  track 
and  compare  users’  traversal  through  the  state  space.  The  resulting  data  can  be  used, 
for  example,  to  check  the  consistency  of  an  estimated  alternative  path’s  cost  to  actual 
usage,  or  to  establish  where  users  may  need  indicators  that  will  help  them  make  better 
choices. 


4.3  Measuring  Optimal  Costs 

The  second  measuring  device  addresses  the  set  of  available  alternative  paths  to 
access  resources  that  a  participant  did  not  use  in  each  situation.  This  allows 
calculation  of  a  form  of  optimal  costs  —  the  costs  of  the  least  expensive  alternative 
path  a  user  could  choose  to  access  a  particular  resource  given  the  current  state  and 
context  the  user  is  actually  in. 
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Context  was  gathered  by  recording  information  snapshots  at  each  new  situation.  In 
our  experiment,  this  occurred  each  time  the  user  started  looking  for  a  resource  and 
each  time  the  participant  opened  a  resource.  The  snapshot  included  extensive  system 
state  information,  such  as  the  current  folder  in  the  File  Open  Dialog  box  and  files 
present  in  My  Recent  Documents,  My  Documents,  the  Recent  Lists  of  each  tracked 
application,  and  so  on.  The  infonnation  snapshots  were  stored  with  the  electronic 
transcript  of  user  behavior  for  concurrent  analysis. 


Given  available  paths  in  context,  optimal  cost  was  computed  by  using  the  model  to 
identify  the  transitions  required  to  reach  the  required  resource.  The  calculations  used 
established  times  for  operations  involving  mental  operations,  keypresses,  and  mouse 
clicks  ([Card  et  al.  1980],  See  Table  2).  For  example,  the  time  taken  to  open  a  file 
using  Start  Menu’s  MyRecentDocuments  as  shown  in  Figure  2  was  calculated  using 
the  GOMS  operators  and  timings  as  follows: 


Optimal  Cost  for  opening  a  file  from  MyRecentDocuments:  M  (Decide  the  path  to 
take)  +  BB  (Click  the  Start  button)  +  M  (Locate  RecentDocuments  in  the  submenu)  + 
P  (Point  with  mouse  on  RecentDocuments)  +  M(look  for  the  file  in  the  submenu)  + 
P(Point  with  mouse  on  file)  +  BB  (Click  on  the  required  file)  =1.2  +  0.2  +  1.2+  1.1  + 
1.2  +  1.1  +  0.2  =  6.2  seconds. 


The  alternative  path  with  the  lowest  cost  was  used  as  the  optimal  cost.  We  cross¬ 
checked  the  achievability  of  these  times  for  alternative  paths,  both  by  comparison  to 
participant  times  when  they  actually  used  them,  and  by  asking  experienced  computer 
users  otherwise  not  involved  in  the  experiment  to  open  resources  using  these  paths.  In 
all  the  cross-checks,  the  fastest  humans  differed  from  optimal  path  cost  calculations 
by  no  more  than  20%.  When  there  were  differences,  the  humans  were  faster.  Thus, 
when  there  are  errors  in  our  optimal  cost  estimates,  it  is  when  the  estimates  are  too 
conservative  (slow).  No  estimate  can  ever  be  perfect,  and  being  overly  conservative 
supports  the  safety  of  our  inferences  better  than  being  too  liberal. 
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4.4  Measuring  Perceived  Costs 

The  third  measuring  device  is  aimed  at  capturing  users’  perceptions  of  two  factors 
of  cost,  and  is  done  via  a  questionnaire.  The  first  was  participants’  self-reported  cost 
assessment  of  the  alternative  paths  from  their  past  experience  with  them,  which  is  our 
operational  definition  of  perceived  costs.  Specifically,  the  questionnaire  included  a  5- 
Likert-scale  question  about  what  their  perceived  cost  to  use  each  alternative  path  was 
(very  easy  to  very  hard). 


In  addition  to  perceived  cost  of  ongoing  usage,  costs  to  learn  to  use  alternative 
paths  may  arise.  To  ascertain  whether  learning  costs  might  play  a  role  in  a  user’s 
overall  perception  of  cost,  the  questionnaire  collected  whether  the  user  was  aware  of 
each  alternative  path  at  all,  or  if  the  user  knew  of  the  alternative  path  beforehand  but 
had  only  infrequently  used  it.  These  costs  are  collected  by  asking  participants  whether 
they  were  aware  that  an  alternative  path  existed  and  their  previous  usage  frequency 
(never  to  always).  These  questionnaires  are  available  in  Appendix  C. 
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5.  EXPERIMENT  DESIGN 

Observational  studies  in  a  controlled  environment  facilitate  the  search  for  causality 
and  relationships  for  situations  in  which  complete  control  is  otherwise  not  possible. 
We  conducted  our  experiment  to  investigate  relationships  between  users’  choices  of 
alternative  paths  and  how  those  choices  related  to  the  cost  types. 

5.1  Participants 

Participants  were  recruited  by  email  to  student  mailing  lists.  The  participants  were 
required  to  have  some  prior  computer  experience,  but  computer  science  majors  and 
students  with  upper-division  coursework  in  computer  science  were  excluded  from 
participation.  Overall,  39  people  (22  women  and  17  men)  took  part  in  the  study,  with 
a  mean  3.28  grade-point  average  and  a  mean  9.09  years  of  computer  experience.  All 
participants  experienced  the  same  treatment. 

5.2  Procedure 

Initially  participants  filled  out  a  pre-session  background  questionnaire  (available  in 
Appendix  C).  Each  participant  was  given  a  common  set  of  resources  (Word 
documents  and  Excel  spreadsheets)  in  a  pre-ordained  folder  hierarchy.  After  filling 
out  the  background  questionnaire,  they  were  then  led  through  an  introductory  session 
(available  in  Appendix  A),  in  which  they  were  instructed  to  browse  through  and  move 
the  files  provided  to  them  so  as  to  gain  some  familiarity  with  their  location  and 
contents.  By  opening  the  files  and  looking  at  their  contents,  participants  also 
populated  “My  Recent  Documents”  in  the  Start  Menu,  and  the  “Recent  List”  in  Word 
and  Excel,  ensuring  that  these  ways  of  opening  files  would  be  available  to  them 
during  the  task. 


In  a  15-minute  session,  participants  then  worked  with  a  practice  task,  which  served 
the  dual  purpose  of  getting  participants  acquainted  with  the  experimental  set-up  and 
of  giving  them  a  purposeful  method  of  further  familiarizing  themselves  with  the 
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content  and  location  of  the  resources.  Participants  then  had  45  minutes  to  perform  the 
main  experiment  tasks. 


Participants  then  fdled  out  a  post-session  questionnaire  (available  in  Appendix  C), 
in  which  they  were  asked  how  they  usually  accessed  resources  in  the  Windows 
environment.  These  questions  were  asked  post-session  rather  than  pre-session  in 
order  to  prevent  the  questions  from  influencing  their  behavior  during  the  experiment. 
Although  this  design  choice  allows  for  the  possibility  of  post-rationalization  by 
participants,  we  considered  this  to  be  the  less  prejudicial  choice.  In  order  to  obtain 
information  about  emotional  costs  (amount  of  mental  and  physical  demand, 
frustration  level,  etc.),  we  used  a  modified  version  of  the  TLX  survey  [Hart  & 
Staveland,  1988]. 


5.3  Task 

The  participants’  task  was  to  answer  up  to  40  questions  (available  in  Appendix  B) 
about  the  U.S.  National  Aeronautics  and  Space  Administration  (NASA).  A  question’s 
role  was  to  represent  a  single  activity  for  a  knowledge  worker;  hence,  questions  were 
designed  to  require  thought,  infonnation  search,  and  infonnation  communication. 


Each  question  required  information  from  one  or  more  of  the  resources  (Word  and 
Excel  files  stored  on  the  participant’s  computer).  Participants  were  instructed  not  to 
use  web  search  to  obtain  the  answers.  In  order  to  maximize  the  need  for  these 
resources,  well-known  facts  about  NASA  were  avoided. 


To  control  for  confounding  effects  and  to  allow  statistical  comparisons  between 
participants  in  terms  of  costs,  each  participant  was  given  a  common  set  of  resources. 
The  resources  were  structured  into  three  folder  levels,  with  seven  top-level  folders,  a 
portion  of  which  is  shown  in  Figure  3.  The  leaves  of  the  tree  contained  50  Word 
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Figure  3:  Folder  hierarchy:  The  folder 
hierarchy  used  in  the  experiment. 

documents  and  5  Excel  spreadsheets.  The  structure  and  names  of  these  resources 
were  devised  by  a  non-technical  end  user,  who  we  recruited  for  this  purpose. 


Questions  were  presented  sequentially  by  software  designed  for  the  experiment 
(Figure  4a  and  Figure  4b).  This  quiz  software  presented  questions  with  an  input  box 
for  typing  in  the  text  answers  or  a  set  of  radio  buttons  for  answering  multiple  choice 


Quiz 

BID® 

File 

28.According  to  Sally  K.Ride's  advice  for  high  school  students  1 
NASA  recruits  people  with  which  of  the  following  backgrounds?  1 

O  History  Of  Science 

O  Microbiology 

C  International  studies 

C  Psychology 

|°K  | 

Figure  4a:  Quiz  software  -  Text  input  question.  4b:  Multiple  Choice  question. 

questions  and  prevented  participants  from  skipping  questions.  The  starting  question 
for  each  participant  was  chosen  at  random. 

We  designed  the  questions  so  that  three  resources  provided  answers  or  partial 
answers  to  approximately  half  the  questions  in  the  experiment.  This  design  choice 
was  based  on  a  set  of  field  visits  with  10  knowledge  workers  (office  workers  and 
educators,  not  software  developers).  The  field  visits  revealed  that  all  these  workers 
had  a  small  set  of  resources,  usually  two  or  three,  to  which  they  refer  repeatedly  and 
frequently.  (The  details  about  the  field  visits  are  available  in  Appendix  F). 
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Knowledge  workers  deal  with  multiple  tasks  and  interruptions.  To  model  this 
aspect  of  the  real  world  in  our  experiment,  certain  questions  (randomly  selected  prior 
to  the  experiment)  were  interrupted  after  the  participant  had  opened  a  predetermined 
number  of  resource  documents.  All  participants  were  interrupted  on  the  same  set  of 
randomly  selected  questions.  When  interrupted,  the  participant  was  forced  to  switch 
to  another  question  (activity).  The  number  of  resources  triggering  a  particular 
question’s  interruption  was  also  randomly  selected  prior  to  the  experiment.  The 
participants  did  not  know  which  questions  would  be  interrupted.  Time  permitting,  the 
interrupted  questions  were  reintroduced  later  in  the  experiment  to  model  the 
interruption  recovery  aspect  of  real-world  tasks  as  well. 


Once  a  participant  had  submitted  an  answer  to  a  question,  Windows  Explorer  (the 
graphical  fde  browser)  and  any  open  resource  documents  (Word,  Excel  etc.)  were 
closed  automatically,  so  that  participants  would  have  to  explicitly  choose  them  again 
if  they  were  needed  later.  The  Word  and  Excel  applications  themselves,  however, 
were  left  running,  to  allow  the  possibility  of  using  the  Recent  List  in  the  File  menu  of 
the  applications  or  the  File  Open  Dialog  box  if  desired  by  the  participant.  The 
automatic  closing  of  applications  and  documents  was  done  to  ensure  that  the  starting 
state  of  each  question  was  the  same.  The  data  was  collected  using  the  TaskTracer 
software  [Dragunov  et  al.  2005]  and  this  is  discussed  in  detail  in  Chapter  4  -  Section 
4.2 
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6.  RESULTS 

6.1  Alternatives,  Choices,  and  Their  Actual  Costs 

Our  state-transition  model  revealed  27  possible  alternative  paths.  Clearly,  the  cost 
of  making  the  decision  from  among  so  many  different  alternative  paths  would  be 
high,  even  if  all  of  these  alternative  paths  were  known  to  the  user. 


This  suggests  that  there  could  be  some  alternative  paths  that  a  user  never  chooses. 
To  understand  to  what  extent  this  occurred,  we  counted  the  alternative  paths  that 
participants  actually  used  during  the  experiment.  As  Table  3  shows,  a  mere  1 1  of  the 
27  possible  alternative  paths  were  used  by  any  of  the  39  participants.  No  participant 
used  any  path  not  represented  by  our  current  state-transition  model. 


Alternatives 

Code 

Participants 

%  of 

Participants 

Windows  Explorer  (existing  window) 

WE 

35 

89.74 

Windows  Explorer  (Address  Bar) 

AB 

16 

41.02 

Windows  Explorer  (My  Computer  on 
Desktop) 

MC 

15 

38.46 

File  Open  Dialog  (application) 

FO 

8 

20.51 

Recent  List  (application) 

RL 

7 

17.94 

Windows  Search  (Windows  Explorer) 

WX 

7 

17.94 

Windows  Explorer  (Start  Menu) 

WM 

6 

15.38 

Windows  Explorer  (folder  shortcut  on 
Desktop) 

WD 

3 

7.69 

Windows  Explorer  (Start  Menu  -  RUN) 

WR 

2 

5.12 

Windows  Search  (Start  Menu) 

ws 

2 

5.12 

My  Recent  Documents  (Start  Menu) 

RD 

1 

2.56 

Table  3:  Alternatives  Used 
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Table  3,  in  top  row  also  shows  how  participants  overwhelmingly  chose  to  access 
resources  through  alternative  paths  involving  Windows  Explorer.  (These  results  also 
mirror  the  proportion  of  individual  resource  accesses  for  which  these  alternative  paths 
were  used.)  Once  they  had  used  Windows  Explorer  and  there  was  an  existing  window 
for  it,  they  kept  returning  to  it  (even  though  it  may  have  been  hidden). 


We  then  looked  at  the  actual  costs  of  the  alternative  paths  chosen  by  the  users.  The 
box  plots  in  Figure  5  show  the  actual  costs  for  each  alternative  path,  calculated  as  per 
the  measuring  device  described  in  Chapter  4-Section  4.2.  As  can  be  seen  from  the 
rightmost  two  box  plots,  searching  was  more  costly  than  other  alternatives.  ANOVA 
analysis  of  all  actual  costs  confirmed  this,  revealing  a  significant  main  effect  on 
actual  cost  of  search  versus  non-search  (/;<(). 000 1 ).  Not  surprisingly  given  the 
participants’  abilities  to  choose  their  alternative  paths,  there  were  also  significant 
differences  among  participants  (/;<(). 000 1 )  and  there  was  a  significant  interaction 
effect  (/?<0.0001). 


RD  RL  WE  FO  AB  WD  MC  WR  WM  WS  WX 


FO  =  File  Open  Dialog  (application) 

RD  =  My  Recent  Documents  (Start  Menu) 

RL  =  Recent  List  (application) 

AB  =  Windows  Explorer  (Address  Bar) 

WE  =  Windows  Explorer  (existing  window) 
WD=Windows  Explorer  (folder  shortcut  on 
Desktop) 

MC=  Windows  Explorer  (My  Computer  on 
Desktop) 

WM  =  Windows  Explorer  (Start  Menu) 

WR  =  Windows  Explorer  (Start  Menu  -  RUN) 
WS  =  Windows  Search  (Start  Menu) 

WX  =  Windows  Search  (Windows  Explorer) 


Figure  5:  Actual  cost  of  each  alternative;  see  Table  1  for  a  legend.  RD  and  RL  are 
"recent  lists",  low  in  cost  but  viable  only  when  they  contained  the  desired  file.  WE 
existed  only  after  a  question’s  first  file  open.  The  other  alternatives  were  always 

available. 
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We  were  interested  in  determining  how  many  of  the  alternatives  they  used  during 
the  experiment  and  how  many  of  those  they  knew  from  before  the  experiment.  Most 
participants  used  2  to  3  alternative  paths  (Figure  6)  during  the  experiment.  The  mean 
number  of  chosen  alternative  paths  per  participant  was  2.64.  In  the  post-session 
questionnaire  (available  in  Appendix  C)  we  obtained  information  about  the 
alternatives  that  they  knew.  We  then  compared  this  against  the  alternatives  that  they 
actually  used.  The  results  are  shown  in  Table  4.  Although  the  table  does  not  yield 
significant  results  (chi-square  test,  p-value=0.1353),  it  is  interesting  to  note  that  90% 
of  what  they  used,  they  already  knew.  Another  interesting  fact  is  that  they  used  only 
20%  of  what  they  knew. 


Number  of  alternatives  used 

Figure  6:  Number  of  alternatives  used  by  participants. 

Recall  that  the  experiment  included  interruptions  because  of  their  presence  in  the 
real  world.  The  results  reported  above  and  throughout  this  paper  include  both 
interrupted  and  non-interrupted  situations.  The  trends  for  interrupted  data  and  non- 
interrupted  data  were  the  same,  and  thus  we  did  not  separate  them  for  analysis. 


Used 

Not  Used 

Knew 

59 

256 

Didn’t  Know 

7 

61 

Table  4  -  Alternatives  Known  and  Used 
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The  network  diagram  in  Figure  7  shows  whether  and  how  often  a  particular 
alternative  was  used  in  conjunction  with  another  alternative.  Nodes  in  the  network 
represent  the  1 1  alternatives  that  were  used  by  participants  in  the  experiment.  An 
edge  between  two  nodes  indicates  that  both  these  alternatives  were  used  in  the 
experiment  session  by  the  same  participant.  The  thickness  of  the  edge  between  two 
alternatives  indicates  the  number  of  instances  in  which  the  two  alternatives  were  used 
together. 


FO  =  File  Open  Dialog  (application) 

RD  =  My  Recent  Documents  (Start  Menu) 

RL  =  Recent  List  (application) 

AB  =  Windows  Explorer  (Address  Bar) 

WE  =  Windows  Explorer  (existing  window) 
WD=Windows  Explorer  (folder  shortcut  on  Desktop) 
MC=  Windows  Explorer  (My  Computer  on  Desktop) 
WM  =  Windows  Explorer  (Start  Menu) 

WR  =  Windows  Explorer  (Start  Menu  -  RUN) 

WS  =  Windows  Search  (Start  Menu) 

WX  =  Windows  Search  (Windows  Explorer) 


Figure  7:  Network  Diagram  -  Combinations  of  alternatives 


This  network  diagram  again  emphasizes  the  importance  Windows  Explorer  had  to 
the  participants  as  a  core  component  for  accessing  resources.  Moreover,  it  reveals  an 
application-centric  approach,  in  that  participants  tended  to  use  File  Open  Dialog  (FO) 
together  with  Recent  List  (RL)  in  that  application.  In  fact,  it  appears  that  participants 
had  certain  “constellations”  of  preferred  strategies.  There  were  also  “anti¬ 
constellations”  —  certain  combinations  of  alternatives  that  were  almost  never  used 
together.  For  example,  My  Recent  Documents  (RD)  was  never  used  in  combination 
with  any  kind  of  Windows  Search  (WS  and  WX),  nor  with  any  desktop  shortcuts 
(MC  and  WD),  nor  with  File  Open  Dialog  (FO). 
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Discussion 

It  appears  that  the  Windows  Explorer  alternatives  were  attractive  to  participants. 
This  is  consistent  with  previous  findings  reporting  that  users  preferred  strategies  with 
interactive  feedback  about  their  status  towards  locating  a  file  [Fu  and  Gray  2004, 
Teevan  et  al.  2004].  Windows  Explorer  excels  in  this  regard  by  continually  keeping 
users  apprised  of  the  context,  which  may  help  to  explain  its  apparent  popularity. 


Note  that  some  alternative  paths  were  available  only  at  certain  points.  For 
example,  locating  a  file  through  My  Recent  Documents  would  only  be  successful  if 
this  resource  had  been  opened  recently  enough  to  feature  in  the  list.  Even  though  the 
actual  cost  for  an  alternative  path  may  be  low,  this  potential  risk  could  play  an 
important  role.  In  the  terms  of  the  Attention  Investment  Model  [Blackwell  2002], 
attempting  such  alternative  paths  involved  a  risk  to  the  participant  that  they  may  not 
receive  the  benefit  of  locating  the  file  using  that  alternative  path.  In  this  event,  yet 
higher  cost  would  be  incurred  by  having  to  backtrack  and  choose  another  alternative 
path. 


Searching  was  not  very  popular.  It  is  possible  that  an  alternative  search  tool  might 
have  been  more  popular  than  the  standard  ones  that  were  present  in  our  experiment. 
Still,  this  result  is  in  agreement  with  previous  research  [Jones  et  al.  2005,  Kaptelinin 
1996,  Teevan  et  al.  2004]  indicating  that  users  in  fact  prefer  manual  browsing  to 
searching.  Note  also  that  the  actual  costs  of  the  other  alternatives  were  much  lower 
(and  less  variable)  than  the  two  alternatives  that  involved  searching,  which  likely 
informed  users’  perception  of  the  downsides  of  searching. 


The  constellations  represent  collections  of  tools  that,  in  sum,  seemed  to  cover 
everything  a  participant  was  trying  to  do.  Having  achieved  this  complete  coverage  of 
all  the  situations  that  were  arising,  a  user  has  less  incentive  to  seek  out  new 
alternatives,  If  so,  this  in  turn  suggests  that  new  ways  that  could  be  developed  may 
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never  or  rarely  be  used,  if  they  do  not  fit  well  in  an  existing  constellation  or  catalyze  a 
new  one. 


6.2  Consideration  of  Optimal  Costs 

How  close  did  participants  get  to  achieving  optimal  costs  (as  calculated  by  our 
measuring  device)?  Optimal  costs  did  appear  to  influence  their  choice  of  alternative 
paths:  there  was  a  significant  relationship  between  actual  and  optimal  user  costs 
(linear  regression,  /K0.01,  F[4,72]=6.695,  R  =0.2711).  Participants  achieved  optimal 
costs  about  38%  of  the  time  in  their  access  paths. 


Notice,  however,  that  optimal  costs  explain  only  about  27%  of  actual  costs.  Much 
of  the  time,  participants  seemed  to  return  repeatedly  to  the  same  alternative  paths, 
even  when  there  were  better  choices.  For  example,  Recent  List  and  My  Recent 
Documents,  when  they  were  available,  could  get  participants  to  the  files  in  only  a 
small  number  of  mouse  clicks  but  instead  other,  costlier  alternative  paths  were  often 
used. 


Discussion 

The  results  show  that,  although  participants  tried  to  a  significant  extent  to  make 
choices  that  were  linked  to  efficiency,  they  did  not  succeed  in  making  the  best  choice 
in  a  large  number  of  cases.  The  reason  for  this  could  be  that  their  perception  of 
alternatives’  costs  was  not  accurate.  The  Attention  Investment  Model  suggests  that 
users  make  decisions  based  on  perceived  costs,  risks,  and  benefits  —  which  may  not 
be  very  close  to  actual  or  optimal  costs.  We  next  investigate  this  possibility  to 
determine  if  their  choice  of  alternatives  was  based  on  their  perceived  costs. 
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6.3  Roles  of  Perceived  Costs 

We  investigated  the  relationship  between  their  perceived  cost  of  alternatives  and 
their  choice  of  alternatives.  We  obtained  the  following  information  from  the 
questionnaire. 

•  Perceived  Cost  -  How  much  work  do  you  think  it  is  to  find  the  right  files  in 
the  following  ways?  [Likert  scale  of  Very  easy-Very  Hard] 

Logistic  regression  is  based  on  probabilities  expressed  as  odds,  and  we  used  it  to 
assess  whether  perceived  costs  or  previous  usage  were  predictive  of  whether  a 
participant  selected  an  alternative.  We  found  that  perceived  cost  (as  measured  by  the 
questionnaire)  of  the  alternative  was  not  predictive  of  the  choice  of  an  alternative. 
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7.  Up-Front  Costs 

Our  results  suggest  that  participants  were  trying  to  make  choices  linked  to  efficiency. 
Even  so,  they  did  not  make  the  most  optimal  choices  and  their  perception  of  costs  had 
no  relationship  to  the  choices  they  made. 


The  costs  that  we  have  looked  at  so  far  are  the  ongoing  perceived  costs.  According  to 
the  Attention  Investment  model,  it  is  important  to  differentiate  between  the  ongoing 
costs  to  get  the  work  done  as  opposed  to  up-front  investments  to  enable  users  to  use 
the  ongoing  method  of  choice.  We  now  turn  our  attention  to  these  up-front  costs  to 
understand  its  role  in  participants’  choices. 


The  Attention  Investment  model  discusses  the  investment  decisions  that  the  users 
need  to  make  when  alternative  courses  of  action  are  available.  These  are  decisions 
that  the  users  have  to  take  initially  while  choosing  an  alternative  path.  Making  the 
right  investment  eventually  leads  to  a  payoff  but  in  order  to  make  this  investment 
users  need  to  spend  mental  and  physical  effort  which  adds  to  their  upfront  costs.  Our 
findings  show  that  participants  were  sensitive  to  these  up-front  costs.  Specifically 
there  are  two  types  of  up-front  costs  that  we  observed.  They  are  the  (1)  up-front  cost 
of  choosing  an  alternative  path  and  (2)  up-front  cost  of  learning  an  alternative  path. 

7.1  Up-front  cost  of  choosing  an  alternative 

The  following  pieces  of  evidence  support  the  importance  of  the  cost  of  choosing  to 
participants.  First,  participants  only  used  a  small  number  of  alternative  paths  during 
the  experiment,  suggesting  that  they  did  not  consider  all  alternatives  before  making  a 
selection.  In  fact,  the  Attention  Investment  Model  suggests  that  users  would  be 
unlikely  to  take  the  time  to  mentally  compare  a  large  number  of  alternatives,  since  the 
costs  spent  in  advance  on  mental  calculations  to  locate  an  optimal  alternative  would 
outweigh  the  cost  saving  gained  from  eventually  carrying  out  an  optimal  alternative 
path.  Second,  participants  knew  about  8  alternatives  on  an  average  but  they  used  only 
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about  2.  Third,  we  can  see  from  the  Table  4  in  Chapter  6  -  Section  6.1  that  they  used 
only  about  20%  of  the  alternatives  that  they  knew.  This  implies  that  those  20%  of  the 
alternatives  covered  “well  enough”  everything  that  they  wanted  to  do.  Recall  also  that 
the  set  of  alternatives  they  used  formed  constellations  as  shown  in  Figure  7  in  Chapter 
6  -  Section  6.1.  These  constellations  apparently  covered  everything  that  they  wanted 
to  do.  This  suggests  that,  in  general,  it  is  often  unlikely  that  users  will  consider  new 
alternatives  because  not  considering  them  reduces  their  up-front  choosing  costs. 


7.2  Up-front  costs  in  learning  new  alternatives 

We  investigated  the  relationship  between  previous  frequency  of  usage  of 
alternatives  and  the  users’  choice  of  alternatives.  We  found  that  the  only  case  in 
which  previous  frequency  of  use  was  predictive  of  an  alternative  being  selected  was 
the  Address  Bar,  which  is  a  component  of  Windows  that  is  not  visible  by  default,  and 
was  thus  very  infrequently  used  by  our  participants.  Its  low  previous  usage  suggests 
that  participants  may  have  previously  expected  up-front  costs  to  leam  to  use  the 
Address  Bar. 


Though  previous  frequency  of  usage  had  no  impact  on  their  choices,  on  the  post¬ 
session  questionnaires,  several  participants  specifically  alluded  to  the  costs  of 
learning  new  alternatives  that  were  not  previously  used.  For  example: 

Q:  Why  do  you  usually  use  these  ways  <the  ways  you  usually  use>? 

"Because  that  is  how  I  learned  and  it  is  easy. " 

"Because  I  am  not  familiar  with  other  ways. " 

"Easy  to  use. " 


The  responses  suggest  satisfaction  with  the  current  perceived  cost-risk-benefit 
ratio  of  prior  investment  in  learning,  implying  no  need  for  additional  learning 
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investment.  Overcoming  this  perception  is  a  challenge  to  new  techniques  aimed  at 
helping  users  to  access  a  desired  resource. 


Additional  evidence  for  participants’  lack  of  enthusiasm  for  bearing  this  learning 
cost  can  be  seen  in  Table  4  and  Used  in  Chapter  6  -  Section  6.1.  Notice  that  about 
90%  of  the  alternatives  that  they  used  were  alternatives  that  they  knew  about  and  10% 
of  alternatives  that  they  used  were  the  alternatives  they  did  not  know  about.  Out  of 
the  7  alternatives  that  were  not  known  but  were  used,  5  of  them  were  the  Address  Bar 
alternative.  The  participants  were  not  willing  to  try  out  new  alternatives  that  they  did 
not  know  about  and  this  suggests  that  there  was  a  consideration  of  learning  cost  in 
their  decision  making. 


Not  spending  time  to  leam  is  a  common  theme  in  studies  reporting  on  similar 
phenomena.  For  example,  it  has  been  observed  that  users  often  use  inefficient 
strategies  even  when  more  efficient  strategies  exist  [Carroll  and  Rosson  1987]  and 
that  inefficient  strategies  are  used  even  by  experts  who  valued  efficiency  [Bhavnani  et 
al.  2000]. 


Learning  time’s  dissuading  power  is  also  consistent  with  the  Attention  Investment 
Model.  It  suggests  that  users  may  be  deterred  by  the  cost  of  learning  new  features, 
even  when  they  realize  that  such  features  may  be  better  ways  to  access  their 
resources,  if  the  up-front  cost  of  learning  is  seen  as  being  too  costly  or  too  risky  an 
investment  of  their  time.  As  with  all  investments,  there  is  a  risk  of  low  or  nonexistent 
return  (such  as  when  a  feature  is  not  helpful  or  the  user  doesn’t  succeed  in  learning), 
thus  potentially  making  the  learning  investment  seem  unattractive. 


7.3  Behavior  Modification  Is  Possible 

Despite  this,  there  was  some  surprising  evidence  of  success  at  encouraging 
learning  new  alternative  paths.  In  designing  this  experiment,  we  chose  to  enable  the 
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Figure  8:  Popularity  of  Alternatives 

Address  Bar  so  that  participants  could  use  that  alternative  if  they  wanted.  We  did  this 
by  making  the  Address  Bar  visible  at  the  bottom  of  the  screen  (in  the  Windows  Task 
Bar)  with  its  initial  contents  pointing  to  a  folder  on  a  local  disk  drive. 

It  turns  out  that  this  design  choice  had  interesting  consequences.  In  our  study,  16 
participants  (41.02%)  used  the  Address  Bar.  The  mean  number  of  file  opens  using 
this  alternative  for  the  16  participants  who  used  it  was  31  (median  36),  which 
amounts  to  37%  (median  43%)  of  the  number  of  opens  they  made.  In  fact,  14  of  these 
16  participants  used  it  for  at  least  18  file  opens.  Further,  participants  not  only  tried 
this  alternative;  they  adopted  it,  making  sustained  use  of  the  Address  Bar  alternative. 


This  is  particularly  interesting  in  light  of  the  fact  that  this  alternative  was  perceived 
to  be  the  costliest.  It  had  been  least  frequently  used  in  the  past  (Figure  8).  5  out  of 
those  16  participants  who  used  it  in  the  study  reported  that  they  did  not  even  know 
that  this  alternative  existed  prior  to  the  experiment.  Clearly,  for  these  5  people,  an 
incentive  was  present  that  was  powerful  enough  to  overcome  resistance  to  new 
learning  costs. 
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Discussion 

The  Address  Bar  may  have  made  participants  curious.  Further,  since  the  Address 
Bar  already  contained  a  default  working  destination,  it  visually  explained  how  to  use 
it,  thereby  implying  low  learning  cost  and  low  risk  experimenting.  Those  who 
followed  up  were  rewarded  for  its  use:  the  necessary  resource  was  accessed.  Given  a 
positive  experience,  they  continued  to  use  it  throughout  the  experiment. 


This  trio  —  arousing  curiosity,  minimal  explanations  encouraging  active 
experimentation  and  leading  to  reward  —  has  led  to  success  in  other  domains  as  well. 
For  example,  Surprise-Explain-Reward  [Wilson  et  al.  2003]  is  a  strategy  that  has  been 
successfully  used  in  software  to  attract  users’  attention  to  unfamiliar  features.  Just  as 
with  the  Address  Bar  case,  it  attempts  to  arouse  the  user’s  curiosity  about  a 
potentially  surprising  new  feature;  he  or  she  can  then  pursue  invoking  an  action- 
encouraging  explanation  and,  potentially,  reap  rewards.  Our  results  corroborate 
empirical  results  of  this  strategy,  suggesting  that  making  a  low-learning  cost,  high- 
reward  alternative  visible  can  motivate  users  to  employ  it,  when  they  otherwise  would 
not  have  even  realized  it  existed. 
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8.  Implications 

8.1  Suggested  Usage  Of  The  Measuring  Devices  By  Researchers 
And  Designers  Of  Task-Centric  Environments 

This  methodology  allows  precise  measuring  of  costs  for  evaluating  and  reasoning 
about  task-centric  user  interface  environments. 

The  state-transition  model :  For  each  available  alternative,  the  state-transition 
model  explicitly  indicates  the  states  that  the  user  must  traverse  and  the  decisions  they 
must  make  in  order  to  achieve  their  goal  of  locating  a  resource.  Because  it  is  an 
analytical  device  only,  it  can  be  used  “stand  alone”,  and  requires  no  technical  set-up 
or  specialized  knowledge.  A  researcher  or  designer  can  use  this  model,  even  without 
the  measuring  devices,  in  the  following  ways: 

•  The  existing  model  can  be  walked  through  to  calculate  order  statistic 
figures  to  users  (i.e.,  number  of  operations)  of  all  the  existing  alternatives. 
These  figures  can  serve  as  estimators  for  the  costs  to  users  of  existing 
alternatives. 

•  Adding  new  alternatives  to  the  state-transition  model  allows  user  costs  of 
the  new  alternatives  to  be  calculated. 

The  first  measuring  device  (actual  user  costs):  This  device  can  be  used  to  gather 
detailed  information  about  what  users  actually  do  and  the  associated  costs.  The  device 
requires  technical  set-up  to  make  sure  all  the  data  is  being  collected  reliably,  and 
specialized  follow-up  in  analyzing  the  data  due  to  its  volume  and  nature.  In  the 
context  of  user  studies,  the  actual  costs  (in  seconds)  of  each  alternative  used  can  then 
be  gathered.  This  can  be  used  to  investigate  “in  usage”  issues.  For  example: 

•  Analysis  of  consistency  of  a  particular  device’s  costs  to  users  in  actual 
usage. 

•  It  provides  a  way  to  track  and  compare  users’  traversal  through  state  space 
with  a  particular  device,  to  see  exactly  how  they  are  using  it  and,  if  they  are 
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not  using  it  wisely,  where  they  may  need  indicators  that  will  help  them 
make  better  choices. 

The  second  measuring  device  (optimal  user  costs):  This  device  brings  the 
theoretical  costs  of  each  available  alternative  (from  the  state-transition  model) 
together  with  the  choices  and  costs  a  user  actually  experiences  in  the  context  of  a  user 
study.  It  uses  state  infonnation  and  the  model  to  find  the  alternatives  available  to  the 
user  that  they  did  not  follow,  and  calculates  the  cost  (in  seconds)  of  these  available 
alternatives.  It  works  in  partnership  with  the  first  measuring  device,  and  likewise 
requires  technical  set-up  and  specialized  follow-up.  The  device  allows  investigation 
of  “best  cost”  issues  such  as: 

•  Theoretical  best-case  costs  derived  from  the  model  alone  to  be  “tested”,  by 
taking  the  number  of  seconds  and  having  real  users  test-drive  those  paths  to 
see  if  they  can  achieve  that  number  of  seconds. 

•  Investigation  of  situations  in  which  users  consistently  deviate  from  better 
alternatives. 

•  Computation  of  which  alternatives  are  best  from  a  user  cost  standpoint, 
given  the  situations  (states)  users  actually  get  into. 

The  third  measuring  device  (perceived  user  costs  and  choices):  This  device  does 
not  require  technical  set-up  or  specialized  follow-up;  it  is  simply  a  questionnaire.  Its 
primary  usage  is: 

•  It  allows  researchers  and  designers  to  obtain  the  user’s  opinion  of  the  costs 
of  the  alternatives  they  know  about.  This  infonnation  may  be  critical, 
because  there  is  some  evidence  that  users’  cost-risk-benefit  decisions  are 
based  in  part  on  these  costs — the  perceived  ones  (which  include  perceived 
learning  costs  and  perceived  cost  of  choosing  an  alternative) — not  on 
actual  costs. 

The  model  and  three  measuring  devices  can  be  used  in  the  context  of  usability 
studies  (as  we  have  already  done)  or  field  studies.  Thus,  it  enables  design  decisions  to 
be  made  based  on  users’  choices  and  their  cost-risk-benefit  analysis  for  actual  and 
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perceived  usage.  It  also  enables  ongoing  evaluation  of  new  alternatives  being 
developed  to  reduce  user  costs. 

8.2  Implications  of  the  Results 

The  results  of  this  experiment  are  a  set  of  findings  with  strong  implications  for 
systems  aiming  to  support  users  in  finding  and  accessing  local  task-specific  resources. 

•  Staying  with  only  a  small  number  of  ways :  Users  tend  to  stick  to  their 
preferred  ways.  (Only  1 1  of  the  27  possible  alternatives  were  chosen  across 
all  participants  during  the  study,  and  almost  all  individual  users  only  used 
two  to  three  different  alternatives.)  This  makes  sense,  because  when  there 
are  a  lot  of  choices  to  consider,  a  user  is  not  likely  to  bear  the  mental  cost 
of  considering  them  all.  This  implies  that  it  will  be  a  challenge  to  attract 
users  to  a  new  alternative. 

•  Constellations  and  anti-constellations :  The  ways  a  user  actually  uses  seem 
to  group  into  “constellations,”  some  of  which  appear  to  be  application¬ 
centric,  and  others  “Explorer-centric.”  For  example,  one  such  constellation 
was  that  users  tended  to  use  Windows  Explorer  or  File  Open  Dialog  (FO) 
together  with  Recent  Fist  (RF)  in  that  application.  There  are  also  “anti¬ 
constellations”:  certain  ways  that  users  never  seem  to  use  together.  The 
constellations  and  anti-constellations  may  help  to  explain  why  participants 
stayed  within  a  small  set  of  alternatives.  This  in  turn  suggests  that 
breakthrough  new  ways  that  might  be  developed  still  may  never  or  rarely 
be  used,  if  they  do  not  fit  well  in  an  existing  constellation  or  catalyze  a  new 
one. 

Things  that  don’t  seem  to  help: 

•  An  alternative ’s  cost  versus  being  selected :  The  ways  that  users  tend  to 
choose  to  access  files  are  not  necessarily  the  most  efficient  ones,  or  even 
perceived  to  be  the  most  efficient  ones.  In  particular,  perception  of  low  cost 
is  not  enough  for  an  alternative  to  join  a  user’s  group  of  favorite  ways. 
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•  Role  of  computer  experience  in  determining  cost:  Users  do  not  get  faster  at 
accessing  local  resources  with  computer  experience:  computer  experience 
had  no  relation  to  the  performance  of  accessing  resources.  This  underlines 
the  importance  of  helping  to  reduce  these  costs,  since  they  are  not  a 
temporary  phenomenon  that  goes  away  after  the  user  gains  experience. 

•  Search  tools:  Searching  was  not  very  popular.  It  is  possible  that  an 
alternative  search  tool  might  have  been  more  popular  than  the  (widespread) 
ones  that  were  present  in  our  experiment,  but  we  believe  that  is  somewhat 
unlikely:  previous  research  has  indicated  that  users  in  fact  prefer  manual 
browsing  to  searching.  Also,  the  actual  costs  of  the  manually  locating 
alternatives  in  our  experiment  were  much  lower  (and  less  variable)  than 
alternatives  that  involved  searching. 

Things  that  do  seem  to  help: 

•  Very  fine-grained  feedback:  It  has  been  established  in  the  literature  that 
users  appreciate  iterative,  interactive  feedback,  and  this  seemed  to  be 
evident  in  their  choice  in  locating  files.  Windows  Explorer  was  very 
attractive  to  participants,  which  is  consistent  with  previous  findings  which 
revealed  that  participants  preferred  strategies  that  gave  interactive  feedback 
about  their  current  status  and  progress  towards  locating  a  file  at  every  step 
along  the  way. 

•  Surprise-Explain-Reward:  It  appears  that  the  perceived  cost  and  risk  of 
learning  new  alternatives  may  have  outweighed  participants’  sense  of  the 
cost  and  benefit  improvements  of  actually  using  them.  This  is  consistent 
with  similar  results  by  others  (the  paradox  of  the  active  user).  However,  it 
appears  to  be  possible  to  modify  behavior  from  a  user’s  pattern,  by 
attracting  them  to  lower-cost  alternatives  through  an  instantiation  of  the 
Surprise-Explain-Reward  strategy.  A  critical  aspect  seems  to  be  to  alert 
users  to  the  presence  of  an  alternative  that  appears  to  have  a  low  learning 
cost  and  risk,  and  which  turns  out  to  also  deliver  low  actual  cost. 
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9.  Conclusion 

In  this  thesis,  we  have  presented  a  methodology  for  investigating  ways  and  costs  for 
users'  local  resource  access,  including  a  detailed  model  of  possible  alternatives  and 
cost  structures.  Using  this  methodology,  we  conducted  an  empirical  study  to  shed 
light  on  the  roles  of  different  kinds  of  costs  on  users’  behavior. 


We  described  three  measuring  devices  that  were  motivated  by  our  methodology, 
which  we  then  instantiated  to  measure  alternatives  and  costs  of  resource  access:  a 
measuring  device  for  collecting  chosen  alternatives  and  their  cost,  a  measuring  device 
for  available  alternatives  and  optimal  cost,  and  a  measuring  device  for  participant- 
reported  alternatives  and  costs. 


The  costs  we  investigated  were  inspired  by  the  Attention  Investment  Model.  We 
learned  that  actual  costs  in  these  circumstances  were  related  to  optimal  costs,  which 
showed  that  users  were  motivated  by  reducing  their  costs  of  day-to-day  operation. 
However,  the  relationship  was  not  strong  enough  to  explain  many  of  their  choices, 
and  in  fact  even  perceived  costs  were  not  significant  predictors  of  users’  choices. 
Instead,  the  amount  and  risk  of  the  up-front  investment  seemed  to  outweigh  the 
ongoing  cost.  This  was  seen  in  participants’  aversion  to  incurring  possible  learning 
costs  and  risks  as  well  as  in  their  avoidance  of  the  up-front  costs  of  considering  a 
large  number  of  alternatives:  they  used  only  two  or  three  favorite  options,  even  when 
they  were  familiar  with  many  more. 


These  results  reveal  that  implicit  assumptions  that  may  underlie  some  of  the  research 
into  improving  these  costs  do  not  hold.  In  particular,  since  our  participants  considered 
only  a  small  fraction  of  the  27  alternatives  possible,  the  evidence  does  not  support  the 
notion  that  users  will  voluntarily  even  consider  such  research’s  emerging  28th 
alternative,  let  alone  learn  how  to  use  it. 
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Thus,  three  challenges  must  be  faced  by  emerging  technological  devices  to  reduce 
resource  access  costs  for  users:  (1)  how  to  make  users  aware  of  these  new  alternatives 
in  a  way  that  does  not  increase  their  already  too-high  up-front  costs;  (2)  how  to 
communicate  low  learning  cost  and  low  risk  of  new  alternatives  —  so  low  that  the 
user  will  be  willing  to  try  them;  and  (3)  how  to  maximize  the  chances  that  a  user’s 
initial  experimentation  with  new  alternatives  will  indeed  deliver  on  its  promise  of  low 
ongoing  cost,  so  that  the  user’s  up-front  costs  will  pay  off. 
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Appendix  A:  Tutorial 


Introduction 

Hi,  my  name  is  Vidya  Rajaram,  and  I  will  be  leading  you  through  today's  study. The 
other  people  involved  in  the  study  are  Dr.Margaret  Burnett,  Dr.Jon  Herlocker, 
Dr. Simone  Stump f,  Kevin  Johnsrude,  Anton  Dragunov,  Julie  Lynn  and  the  assistants 
helping  me  out  today. 


I  will  be  reading  through  this  script  so  that  I  am  consistent  in  the  information  that  I 
provide  you  and  the  other  people  taking  part  in  the  study  for  scientific  purposes. 


The  purpose  of  this  study  is  to  investigate  the  patterns  that  users  follow  when  they 
interact  with  computers,  and,  in  particular,  to  understand  the  cost  (in  terms  of  low- 
level  user  actions  as  well  as  more  high-level  costs  such  as  cognitive  operations)  of 
activities  that  can  be  reduced  or  avoided  with  Task  Tracer  —  the  software  package 
developed  by  our  research  group. 


For  today's  experiment  I  will  lead  you  through  a  brief  tutorial  after  which  you  will 
take  a  few  experimental  tasks.  Please  do  NOT  discuss  this  study  with  anyone.  We  are 
doing  later  sessions  and  we  would  prefer  the  students  coming  in  not  to  have  any 
advance  knowledge. 


Anybody  who  hasn’t  returned  the  Informed  Consent  form  and  the  Background 
Questionnaire,  please  raise  you  hand. 


(After  they  raise  hands) 

Please  fill  the  forms  and  raise  you  hand.  The  assistants  will  collect  it  from  you. 
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If  you  have  any  questions,  please  contact 
Contact: 

-  Dr.Margaret  Burnett  bumett@eecs.oregonstate.edu 

Any  other  questions  maybe  directed  to  the  IRB  Coordinator,  Sponsored 
Programs 

Office,  OSU  Research  Office, (54 1)737-8008 

The  contact  addresses  are  available  in  the  “Informed  Consent”  form  that  you  have 
with  you. 


Tutorial 

Before  we  begin,  I  would  like  to  tell  you  not  to  check  personal  emails  or  work  on 
anything  else  that  is  not  related  to  the  experiment.  Also,  do  not  use  Internet  explorer 
or  any  other  browser.  And  please  raise  your  hand  if  you  have  any  problems  and  the 
assistants  will  help  you  out. 


As  we  go  through  this  tutorial,  I  want  you  to  ACTUALLY  PERFORM  the  steps  I’m 
describing.  Erin  will  be  demonstrating  in  the  screen.  So  if  you  have  any  doubts, 
please  look  at  the  screen. 


Now  I  will  explain  to  you  how  the  experiment  is  structured.  There  are  some  files  in 
your  system  related  to  NASA  (National  Aeronautics  and  Space  administration). 


There  will  be  a  folder  called  NASA  available  on  your  system  in  the  location 
“C:\temp\Documents”.  This  location  is  available  in  the  “location  sheet”  that  has  been 
handed  out  to  you.  Now  let  us  go  to  that  location  to  view  the  folder.  Please  go  to  the 
location  “C:\temp\Documents”.  Here  you  can  see  the  folder  NASA. 


Now  open  that  folder.  (Wait  for  them  to  click) 
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You  can  see  that  this  folder  contains  a  number  of  subfolders  like  “Budgets”,  ”Fun  for 
Kids”,  “Missions”  etc. 

(Erin  will  just  mouse  over  the  folders) 

In  the  folder  “Budgets”,  you  can  find  information  related  to  Budgets  of  NASA.  In  the 
folder  “Fun  for  Kids”,  you  can  find  information  about  what  kids  events  are  being 
organized  in  NASA.  Similarly  all  other  folders  contain  infonnation  about  NASA. 


Now  Let  us  move  these  folders  to  different  locations.  I  want  you  to  move  the  folders 
to  the  locations  I  ask  you  to.  Please  don’t  copy  them  but  move  them  to  the 
destination.  The  location  sheet  which  you  have  contains  the  destinations  to  which  you 
need  to  move  the  folders  to. 


For  example,  move  the  folder  “News-Events-Features”  at  the  location 
“c:\temp\Documents\NASA”  to  the  location  “c:\temp\Documents\Things 
going  on  at  NASA”. 


Please  raise  you  hands  after  you  finish  moving  the  folders. 

❖  Move  folder  “c:\temp\Documents\NASA\News-Events-Features”  to 
“c:\temp\Documents\Things  going  on  at  NASA”. 

❖  Move  folder  “c:\temp\Documents\NASA\Missions”  to 
“c:\temp\Documents\Things  going  on  at  NASA”. 

❖  Move  folder  “c:\temp\Documents\NASA\People  of  NASA”  to 
“c:\temp\Documents\Jobs  at  NASA”. 

❖  Move  folder  “c:\temp\Documents\NASA\Want  to  be  a  part  of  NASA”  to 
“c:\temp\Documents\Jobs  at  NASA”. 
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❖  Move  folder  “c:\temp\Documents\NASA\NASA  Facts”  to 
“c:\temp\Documents\Interesting  F acts”. 

❖  Move  folder  “c:\temp\Documents\NASA\Outerspace  Facts”  to 
“c:\temp\Documents\Interesting  F acts”. 

❖  Move  folder  “c:\temp\Documents\NASA\Fun  for  kids”  to 
“c:\temp\Documents\Things  to  do  at  home  with  your  kids”. 

❖  Move  folder  “c:\temp\Documents\NASA\Proposals”  to 
“c:\temp\Documents\NASA  Grants”. 

❖  Move  folder  “c:\temp\Documents\NASA\Budgets”  to 
“c:\temp\Documents\Where  NASA’s  money  goes”. 

(Assistants  will  check  if  they  moved  the  folders  and  delete  the  NASA  folder). 

Now  lets  go  to  “C:\temp\Documents\Where  NASA’s  money  goes”  .  Here  you  can 
see  a  folder  called  “Budgets”.  Please  open  the  folder  “Budgets”. 

(Erin  should  have  the  folder  open  already) 

This  folder  contains  some  Microsoft  Excel  files  02-06.  These  files  refer  to  the  budget 
information  of  NASA  in  the  different  years.  Now,  please  open  the  file  “04.xls”,  (wait 
for  them  to  open.  Erin  will  already  have  this  open)  you  can  see  infonnation  about 
how  much  budget  was  requested  in  the  year  2004  for  the  different  space  operations. 

(Erin  will  have  the  file  open  and  she  will  mouse  over  the  contents) 

Now  please  close  the  file  “04.xls”.  (Wait  for  them  to  close  the  file) 

Similarly  you  can  find  different  information  about  NASA  in  the  various  folders. 

Now  please  take  a  couple  of  minutes  to  browse  through  the  files  in 
“C:\temp\Documents”  to  familiarize  yourself  with  the  NASA  content. 


46 


(Wait  a  couple  of  minutes  for  them  to  browse  through  the  folders.  No 
demonstration  required). 


Practice  Task 

Now  let  us  go  on  to  the  practice  task.  In  this  task,  you  will  be  presented  with  a  set  of 
questions  one  after  the  other.  Now  please  look  at  the  screen.  You  will  see  a  quiz  box 
like  this.  This  quiz  box  has  a  question  and  a  text  box  for  you  to  type  the  answer.  Some 
questions  will  be  multiple  choice  and  some  questions  will  require  you  to  type  in  the 
answer  in  the  text  box  provided.  Once  you  type  the  answer  you  will  have  to  hit  the 
OK  button.  You  cannot  pass  any  question. 

(Erin  will  have  this  open  already  and  will  bring  up  the  question  popping  box.  Erin 
will  also  have  the  answer  file  open) 

All  the  questions  are  about  NASA.  Your  goal  is  to  answer  the  questions  correctly. 
The  answers  to  ALL  the  questions  can  be  obtained  from  the  files  in 

“C:\temp\Documents”. 


For  example,  in  this  question  “How  many  activities  are  available  for  kids?”  This  is 
where  Erin  got  the  answer  from.  Type  in  the  answer  in  the  text  box. 

(Erin  shows  the  file  and  types  in  the  answer  in  the  question  popping  box) 

So  after  you  answer  the  question,  please  click  on  the  “OK”  button.  Once  you  hit  the 
“OK”  button,  some  windows  and  the  some  documents  that  were  open  will  be  closed 
automatically. 


For  example,  please  look  at  the  screen. 

(Have  some  windows  open  already  so  that  we  don  V  create  any  bias  in  the  way  the 
file  is  opened.  Erin  will  click  on  the  OK  button). 
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The  answers  to  most  of  the  questions  will  require  information  from  multiple  Word 
documents  and  Excel  workbooks.  So  you  might  have  to  look  up  multiple  files  to 
answer  a  single  question. 


Take  you  own  time  to  answer  the  questions.  Please  try  to  answer  as  many  questions 
as  possible  correctly. 


Now  the  assistants  will  bring  up  the  quiz  box  for  you.  Please  DO  NOT  start  until  I  ask 
you  to. 

(Assistants  will  bring  up  the  question  popping  box  and  tell  them  not  to  close  the 
command  prompt) 

Now  you  can  start  you  practice  tasks.  I  will  tell  you  when  you  need  to  stop. 

(About  10  minutes) 

Now  we  are  finished  with  the  practice  task. 

Assistants  will  now  close  down  the  quiz  box. 

(Assistants  will  close  down  the  question  popping  s/w  and  take  away  the  ‘ cheat 
sheet’) 


Main  Task 

Let  us  proceed  to  the  main  experiment.  This  will  be  similar  to  the  practice  task.  You 
will  be  presented  with  a  set  of  questions  one  after  the  other.  Y ou  will  have  to  find  the 
answers  to  these  questions  from  the  files  in  the  location  “C:\temp\Documents”. 


You  might  need  to  look  up  multiple  files  in  order  to  answer  one  single  question.  You 
cannot  pass  any  questions.  After  answering  the  questions  hit  on  the  “OK”  button. 
When  the  OK  button  is  clicked,  some  of  the  open  windows  and  documents  will  get 
closed  down  automatically. 
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Interruptions 

There  are  some  questions  on  which  you  will  be  interrupted.  That  is  while  working  on 
certain  questions,  a  new  question  will  be  popped  up  preventing  you  from  finishing  the 
question  you  had  been  working  on.  When  the  new  question  pops  up,  some  of  the 
windows  and  some  documents  that  are  opened  will  be  closed  automatically.  These 
interrupted  questions  may  or  may  not  appear  again  in  the  quiz. 


I  would  like  to  remind  you  again  not  to  check  any  personal  emails  or  work  on  any 
other  stuff  not  related  to  the  experiment.  As  I’ve  said  before,  do  not  use  Internet 
explorer  or  any  other  browser.  Do  not  use  web  search  to  obtain  answers  for  the 
questions.  Answers  to  all  questions  can  be  obtained  from  the  files  in  the  location 
“C:\temp\Documents”. 

The  assistants  will  bring  up  the  quiz  box  for  you.  Please  DO  NOT  start  until  I  ask  you 
to. 

(Assistants  will  bring  up  the  question  popping  software  and  tell  them  not  to  close 
command  prompt). 

So  now  you  can  start  the  main  task.  I  will  tell  you  when  you  need  to  stop. 

(Experiment  Begins) 

(About  40  minutes) 

(Experiment  ends) 

The  main  task  is  over. 

Post-Session  Questionnaire 

(Hand  out  Post-session  questionnaire  and  ask  them  to  fill  it  out). 

Now  I  would  like  you  all  to  fill  out  the  questionnaire  that  is  being  handed  out  to  you. 

(Collect  the  questionnaire  back).  This  should  roughly  take  about  10  minutes.  Please 
raise  your  hand  once  you  have  finished  and  the  assistants  will  collect  the  fonns  from 


you. 
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Please  DO  NOT  touch  the  computer  anymore. 

(After  collecting  all  forms) 

Now  you  will  need  to  sign  the  receipt  and  collect  your  money  from  Simone. 

(Hand  out  receipt  and  pay  them). 

Thank  You 

I  would  like  to  thank  you  all  for  participating  in  the  experiment  and  being  very 
cooperative.  I  will  be  very  happy  to  answer  any  questions  you  have.  Once  again  I 
would  like  to  thank  you  all  on  behalf  of  the  Task  Tracer  team. 
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Appendix  B  -  Tasks 

The  following  questions  were  given  to  the  participants  as  part  of  a  quiz  task.  The 
answers  were  to  be  obtained  from  a  set  of  Word  documents  and  Excel  spreadsheets 
provided  to  them. 

1 .  What  is  the  nickname  of  the  flight  director  who  is  standing  second  from  right  in  the 
back  row  in  the  group  photo? 

Answer:  Bob 

2. How  much  budget  was  requested  for  Lunar  Exploration  in  2005? 

Answer:  70 

3.  Where  will  the  national  space  day  celebration  be  held? 

Answer:  Virginia  at  the  National  Air  and  Space  Museum's  Steven  F.  Udvar- 
Hazy  Center 

4.  In  the  flight  directors  group  photo  there  is  a  person  sitting  second  from  right  in  the 
front  row.  In  which  mission  centre  is  her  individual  photo  taken? 

Answer:  Shuttle  (White)  Flight  Control  Room  in  Houston's  Mission  Control 
Center. 

5.  On  which  date  was  the  news  'hubble  celebrates  15th  anniversary'published? 

Answer:  04.25.05 

6.  There  are  8  parts  in  the  grant  proposal.  What  is  the  3rd  part  about? 

Answer:  Project  Description 

7.  There  is  an  individual  photo  of  a  flight  director  whose  last  name  is  Ridings.  Where 
is  he/she  in  the  group  photo?  Specify:  row  -  number  -  direction. 

Answer:  front_2nd  left 

8.  How  much  budget  was  requested  for  Biological  and  Physical  research  in  2004? 


Answer:  973 
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9.  In  the  memory  game  for  kids  where  do  the  two  friends  go? 

Answer:  Saturn 

10.  On  which  date  was  the  flight  directors’  group  photo  taken? 

Answer:  1 1  February  2005 

1 1.  In  which  year  was  the  Budget  requested  for  SOLAR  SYSTEM  EXPLORATION 
the  greatest? 

Answer:  2004 

12.  What  is  the  name  of  the  kid  who  did  the  deep  impact  coloring  page? 

Answer:  Diana 

13.  For  the  vehicle  launched  on  4th  May  how  many  configurations  were  taller  than 
70m? 

Answer:  1 

14.  How  old  was  the  boy  who  wrote  the  poem  ’The  boy’s  first  flight'? 

Answer:  9 

15. In  which  year  was  the  budget  requested  for  Education  the  highest? 

Answer:  2005 

16.  What  is  the  name  of  the  spaceship  in  one  of  the  kid's  stories? 

Answer:  Atlantis 

17.  Administrative  and  Management  constitute  what  %  of  Nasa’s  positions? 

Answer:  24 

18.  There  are  8  parts  in  the  grant  proposal.  What  is  the  6th  part  about? 

Answer:  Facilities  and  equipment 

19.  The  individual  photos  of  how  many  flight  directors  are  available? 

Answer:  3 

20.  When  was  NASA  started? 


Answer:  1958 
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21.  Was  the  budget  requested  for  Constellation  systems  greater  for  2004  than  in 
2005? 

Answer:  yes 

22.  How  many  flight  directors  are  standing  in  the  group  photo? 

Answer:  5 

23.  In  which  year  did  Human  Research  Technology  require  maximum  budget? 

Answer:  2005 

24.  The  research  news  on  02/08/05  is  about  a  conference.  Who  opened  this 
conference? 

Answer:  Dr.  Adena  Loston 

25.  The  activity  taught  by  a  girl  required  how  many  items? 

Answer:  1 1 

26.  How  many  days  after  the  flight  directors’  group  photo  was  taken  were  the 
individual  photos  taken? 

Answer:  14 

27. How  much  budget  was  requested  for  Mars  Exploration  in  2005? 

Answer:  690.9 

28. According  to  Sally  K.Ride’s  advice  for  high  school  students  NASA  recruits  people 
with  which  of  the  following  backgrounds?  a.History  Of  Science,  b.Microbiology, 
c. International  studies,  d.Psychology 

Answer:  Microbiology 

29.  How  many  questions  are  there  in  the  'Questions  and  Answers'? 

Answer:  1 1 

30. NASA  uses  a  tool  for  recruting.  When  was  the  company  that  founded  this  tool 
started? 

Answer:  1988 

31.  How  much  budget  was  requested  for  Kepler  development  in  2005? 
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Answer:  127.2 

32. How  many  MEN  are  there  in  the  flight  directors  group  photo? 

Answer:  6 

33. In  which  year  was  the  budget  requested  for  Space  science  the  greatest? 

Answer:  2004 

34.1n  Nasa's  team  who  is  the  Chief  Financial  Officer? 

Answer:  Gwendolyn  Sykes 

35. In  one  of  the  activities  a  boy  wrote  the  story  Going  to  NASA.  According  to  the 
story  what  had  fallen  on  him  in  the  middle  of  the  night? 

Answer:  bulletin  board 

36. When  is/was  the  Earth  System  Science  Fellowship  proposal  due? 

Answer:  01-Jun-2005 

37.  There  is  a  flight  director  whose  individual  photo  has  been  taken  at  Shuttle  (White) 
Flight  Control  Room  in  Houston's  Mission  Control  Center.  Where  is  he/she  standing 
in  the  group  photo?  Specify:  row  -number  -direction. 

Answer:  back-3rd-right 

38.  In  which  year  was  the  budget  requested  for  Aeronautics  technology  the  smallest? 


Answer:  2006 
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Appendix  C  -  Questionnaires 


BACKGROUND  QUESTIONNAIRE 


1.  Subject  Number 


2.  Gender  M  F 

3.  What  is  your  Major? 


4.  Year  or  degree  completed 


Freshmen 
^  Sophomore 
Junior 
Senior 
PostBac 

Graduate  student 


5 .  Cumulative  GPA  (roughly) 


6.  IS  ENGLISH  YOUR  PRIMARY  LANGUAGE  F  Yes  F  No 


If  not,  how  long  have  you  been  speaking  English? 


years 
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7.  What  have  you  used  computers  for?  (please  check  all  that  apply) 


r 

For  high  school 

How  many  years? 

r 

For  college 

How  many  years? 

r 

For  professional  use 

How  many  years? 

r 

For  personal  use 

How  many  years? 

8.  Do  you  own  a  computer?  Yes  No 


9.  Do  you  use  university  computer  labs?  Yes  ^  No 


10.  Which  do  you  use  more?  Own  Computer  ^  University  computer 


1 1 .  What  account  do  you  use  most  frequently  to  access  university  computers 
(e.g.,  ONID,  ENGR,  Forestry  etc.)? 


12.  Which  campus  computer  lab  do  you  use  most  often  (e.g.,  Milne,  Hovland, 
Forestry, 

Business,  etc.)? 
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Subject  Number: 

POST  SESSION  QUESTIONNAIRE 

1.  Please  check  any  one  of  the  following: 


Low  High 

1  1 

MENTAL  DEMAND 

How  much  mental  and  perceptual 
activity  (e.g  thinking,  deciding, 

calculating,  remembering,  looking, 

searching  etc.)  was  required  to  answer 
the  questions? 

L 

L 

L 

L 

L 

TEMPORAL  DEMAND 

How  much  time  pressure  did  you  feel 
due  to  the  rate  or  pace  at  which  the 
questions  appeared? 

L 

L 

L 

L 

L 

PERFORMANCE 

How  successful  do  you  think  you  were  in 
answering  the  questions? 

L 

L 

L 

L 

L 

EFFORT 

How  hard  did  you  have  to  work 
(mentally  or  physically)  to  accomplish 
your  level  of  performance? 

L 

L 

L 

L 

L 

FRUSTRATION  LEVEL 

How  insecure,  discouraged,  irritated, 
stressed  and  annoyed  versus  secure, 
gratified,  content,  relaxed  and 

complacent  did  you  feel  while  answering 
the  questions? 

L 

L 

L 

L 

L 
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2.  How  do  you  usually  get  to  files  in  your  system? 


3.  Besides  the  ones  you  listed  above,  do  you  ever  use  any  other  ways?  If  yes,  what 
are  they? 


4.  Why  do  you  usually  use  the  ways  you  gave  in  Question  2? 


5.  What  helps  you  remember  where  your  files  are  in  the  system? 


6.  Which  was  more  difficult?  (Choose  one) 


^  Getting  the  right  file 

Finding  the  information  within  the  file 


7.  How  do  you  store  files  that  belong  to  more  than  one  topic? 

(For  example,  if  you  have  payroll  information  about  students,  would  you  store  it 
in  Payroll  folder,  students  folder,  or  both,  or  some  other  way?) 
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8.  Are  you  familiar  with  the  following  ways  to  access  files  in  MS  Windows 
environment? 


O  My  Documents 
IQ  My  Pictures 


£  Body, java 
lid]  Calendar2004.doc 
lid]  Calendar2005.doc 
I  SI]  6radBuddyRoles.doc 
GrayScale.doc 

iiid]  HealthInsurancePlans.doc 
ii]  IUI-2005_13.doc 
Iiid]  Lafindubal.doc 
iid]  Phone Directory.doc 
IB  Platform. java 
lid]  Puzzles.doc 
lid]  Rent_EMW.doc  l 

lid]  Seattle-Tacoma.doc 
j  i|d]  stumpf  _aaai2005_draf  t3 ,  doc 
Ht  WS405StumpfS.pdf 


Folders 


Drojects\NA5A 


Date  Modified 


Q  Budgets 
tj)Fun  for  kids 
Q  Missions 
Q  NAS  A  Facts 
Q  News-E  vents-Features 
QOuterspace  Facts 
Q  People  of  NASA 
Q  Proposals  —  ■  — 

Q  Want  to  be  part  of  NASA 


File  Folder  19.05.2005  15:02 

File  Folder  19.05.2005  15:02 

File  Folder  19.05.2005 15:03 

File  Folder  19.05.2005 15:02 

File  Folder  X”’" 

File  Folder  Navigate  fc 

File  Folder  ....  . _ c 


ffl  O  expat 
SI  |Q  math 
S)  Q  MySQL.NET 
0  Q  NASA 
S  Q  Budgets 
a  Q  Fun  for  kids 
B  Q  Missions 
a  Q  NASA  Facts 
a  Q  News-Events-Features 
B  Q  Outerspace  Facts 
B  Q  People  of  NASA 
B  Q  Proposals 
a  Q  Want  to  be  part  of  NASA 
aQ  TaskTracer 


Windows  Update 


:9  objects  (Disk  free  space:  165  GB) 


Address  Bar 

□  Yes  □  No 


Q  C:\Documents  and  Settings 


My  Documents  folder 

□  Yes  □  No 


Search  Tool 

□  Yes  □  No 


P 


Favorites  List 

□  Yes  □  No 


My  Recent  Documents 

□  Yes  □  No 


Navigate  folders  in 
Windows  Explorer 

□  Yes  □  No 


Settings 

Search 

Help  and  Support 

-F)  Run"' 


Quick  Launch  Bar 

□  Yes  □  No 


Links  Toolbar 

□  Yes  □  No 


Z:\projects\NASA 


Figure  la:  Desktop  Screenshot  -  Familiar  Ways 


Figure  lb:  Application  Snapshot  -  Familiar  Ways 
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9.  How  much  work  do  you  think  it  is  to  find  the  right  files  in  the  following  ways 
(see  picture)? 


||Q  My  Document; 

Q  My  Pictures 

Body. java 
Calendar2004.doc 
Calendar2005.doc 
GradBuddyRoles.doc 
§J  GrayScale.doc 
8)  Health  Insurance  Plans.doc 
l§]  IUI-2005_13.doc 
8]  Lafindubal.doc 
8]  PhoneDirectory.doc 
j£  Platform.java 
8]  Puzzles.doc 
8]  Rent_EMW.doc 
8]  Seattle-Tacoma.doc 
18]  stumpf_aaai2005_draft3.doc 

Quick  Launch  Bar 

□  Very  easy 

□  Easy 

□  Neutral 

□  Hard 

□  Very  hard 

□  Don't  Know 


^My  Documents  folder > 

□  Very  easy 

□  Easy 

□  Neutral 

□  Hard 

□  Very  hard 

□  Don’t  Know 


a  / 

□  Very  easy 

Ji3e3 

□  Easy 

*] 

Back  ’  1 Y  y  Search 

□  Neutral 

□  Hard 

□  Very  hard 

Don’t  Know  ^ 

\  |li;|T  :  Links  Icl Google 

Addre;;.  Q  Z:\projects\Nl  \ 

v  flGo 

r"5  f  Favorites  List'Y  1  N 

ze  Type  Date  Modified 

My  Recent  Documents 

□  Very  easy 

□  Easy 

□  Neutral 

□  Hard 

□  Very  hard 


□  Very  easy 

□  Easy 

□  Neutral 

□  Hard 

□  Very  hard 
Don’t  Know  ^ 

+  Q  News-Events-Features 
ffi  P1  frersnarp.  Farh: 

t  Address  Bar 

□  Very  easy 

□  Easy 

□  Neutral 

□  Hard 

□  Very  hard 

□  Don’t  Know 


IQ  Fun  for  kids 
Q  Missions 
QNASA  Facts 
QNews-Events-Features 
QOuterspace  Facts 
Q  People  of  NASA 
Q  Proposals 

QWant  to  be  part  of  NASA 


File  Folc 
File  Folc 
File  Folc 
File  Folc 
File  Folc 


Q  C:\Documents  and  Settings 


C Links  Toolbar 

□  Very  easy 

□  Easy 

□  Neutral 

□  Hard 
Very  hard 
Don’t  Know 


Navigate  folders  in 
Windows  Explorer 

□  Very  easy 

□  Easy 

□  Neutral 

□  Hard 

□  Very  hard 

□  Don’t  Know 


Local  intranet 


Google  E-mails.xls  Notes.doc  Q  Homepage 


Figure  2a:  Desktop  Snapshot  -  Perceived  Cost 


Figure  2b:  Application  Snapshot  -  Perceived  Cost 
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10.  How  frequently  do  you  use  these  different  ways  to  get  to  files  in  MS  Windows 
environment? 


My  Documents  folder 

□  Never 

□  Seldom 

□  Sometimes 

□  Often 

□  Always 

- r  Platform. java 

Windows  Update  Puzzles.doc 

Rent_EMW.doc 
» lj|]  Seattle-Tacoma.doc 
■  iS]  stumpf _aaai2005_draft3 .  dcj 
fT-  W5tQ55tumaf5.pdf 

Quick  Launch  Bar 

□  Never 

□  Seldom 

□  Sometimes 

□  Often 

□  Always 


Search  Tool 

□  Never 

□  Seldom 

□  Sometimes 

□  Often 

□  Always 


n  a  i _ _ 


IS  £3  expat 
S  Q  math 
ffl  Q  MySQL.NET 
B  Q  NASA 
IS  Q  Budgets 
IS  Fun  for  kids 

IS  Missions 

IS  |£»  NASA  Facts 
IS  News-Events-Features 

IS  £)  Outerspace  Facts 
E  People  of  NASA 

IB  £j  Proposes _ 

s  &WY  a 


Q  TaskTr 


Address  Bar 

□  Never 

□  Seldom 

□  Sometimes 

□  Often 

□  Always 


Name 

O  Budgets 
QFun  for  kids 
Missions 
Q  NASA  Facts 
JNews-Events-Se 
Outerspace  Facts 
Q  People  of  NASA 
Q  Proposals 

QWant  to  be  part  of  NASA 


0Go  Links  @  Google  SJ  E-mails. xls  Notes.doc  Q  Homepage 


Figure  3a:  Desktop  Snapshot  -  Frequency  of  Use 


Figure  3b:  Application  Snapshot  -  Frequency  of  Use 
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11.  Please  check  any  one  of  the  following,  considering  the  interruptions. 


Low  High 


MENTAL  DEMAND 

How  much  mental  and  perceptual  activity 
(e.g  thinking,  deciding,  calculating, 

remembering,  looking,  searching  etc.)  I-  I-  I-  I-  I- 

was  required  when  you  were  interrupted? 


FRUSTRATION  LEVEL 

How  insecure,  discouraged,  irritated, 
stressed  and  annoyed  versus  secure, 
gratified,  content,  relaxed  and 
complacent  did  you  feel  when  you  were 
interrupted? 


r  r  r  r  r 


THANK  YOU 
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Appendix  D  -  State  Diagrams 


1.  Choose  another  alternative 


Figure  4:  Choose  an  Alternative  Path 


2.1  Desktop  chosen 


Figure  5a  -  Desktop  Chosen 
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2.2  Existing  Windows  Explorer  chosen 


i 

' 

> 

,  Select  icon 

f  \ 

3.3  Open  file  from 

Decide  _ 

1.  Choose  another 

Windows  Explorer 

alternative 

Figure  5b:  Existing  Window  Explorer  Chosen 


2.3  Address  Bar  chosen 


Figure  5c:  Address  Bar  Chosen 
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2.4  Quick  Launch  Bar  chosen 


(5) 


Figure  5d:  Quick  Launch  Bar  Chosen 


2.5  Links  Toolbar  chosen 


Figure  5e:  Links  Toolbar  Chosen 
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2.6  Existing  Application  Window  chosen 


1 

> 

_ 

,  Select  icon 

3.4  Open  file  from 

Decide  _  1.  Choose  another 

[  Application 

wi 

alternative 

l  1 

Figure  5f:  Existing  Application  Window  Chosen 


2.7  Start  Menu  chosen 


^  Select  icon 

Start  Menu 
activated 


Navigate 


“All  Programs” 
visible 


Select  icon 


^ _ 

Navigate 

1  \  _ 3 

Navigate 

[ _ 

Navigate 

\  ( 

Navigate 

- \  t - 

Navigate 

\  ( 

Navigate 

"V  (  1 

My  Documents 


visible 
Select  icon 


{Application 
visible 


Select  icon 


Search 

visible 


My  Recent  Documents  My  Computer 


visible 


in  focus 


“RUN” 

visible 


Select  icon 


sC 


Select  icon 


Select  icon 


Select  icon 


2.8  Open  file  using  3.2  Open  file  from  3.4  Open  file  from  35*0penflle 
“All  Programs”  My  Documents  Application  usjn^  Search 


.1  Open  file  from 
My  Recent 
Documents 


3.3  Open  file  from 
Windows  Explorer 

J/  V  J 


2.9  Open  file 
using  “RUN”! 


Decide 


( - - 

1.  Choose  another 

alternative 


Figure  5g:  Start  Menu  Chosen 
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2.8  Open  Application  from  “All  Programs” 


t 

> 

Navigate 

J  Menu  visible 

A 

Navigate  i  r  Select  icon 

Menu  opened 

r  Navigate 

Decide  ^  j  Choose  another 

1  Application  visible 

alternative 

l _ / 

_ ' 

,  Select  icon 

r  i 

3.4  Open  file  from  Application 

Figure  5h:  Open  Application  from  “All  Programs” 


2.9  Open  file  from  “RUN” 


Figure  5i:  Open  File  from  “RUN” 
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3.1  Open  file  from  My  Recent  Documents 


Figure  6a:  Open  File  from  My  Recent  Documents 


3.2  Open  file  from  My  Documents 


Figure  6b:  Open  File  from  My  Documents 
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3.3  Open  file  from  Windows  Explorer 


Figure  6c:  Open  File  from  Windows  Explorer 
3.4  Open  file  from  Application 


Select  icon 


File  menu  opened 


, ,  Navigate 


Menu  items  visible 


Navigate  , , 


Select  icon 


Select  icon 


File  Open  Dialog  Box  opened 


Recent  List  visible 


f - 3 

r 

Navigate 

v — j 

1 

|  Document  visible 

Navigate 

V 

^  Navigate 


Folder  /  Icons  in 


Navigate 


Select  icon 


Select  icon 


Folder  opened 


Decide  Jl.  Choose  another 
alternative 


Figure  6d:  Open  File  from  Application 


69 


3.5  Open  file  using  Search 


lr  Select  icon 

Search  text  box  visible 


1  Input  text 
Search  text  entered 


_ 3 

,  Select  icon 

Decide 

1.  Choose  another 

Search  Results  displayed 

- ► 

alternative 

_ 1 

Navigate 

Document  visible 
Select  icon 


♦ 


Figure  6e:  Open  File  using  Search 


Action  Grammar 

Decide  -  M 
Navigate  -  (MP)* 

Select  icon  -  L  |  L  E  |  R  (P*  L  |  K*)  |  D  |  K*  (D  |  E  |  L  |  L  E  |  R  P*  L  |  R  K*) 
Input  text  -  K* 


Key 

M  -  Mental  operation 
P  -  Mouse  movement 
L  -  Left-click 
E  -  Enter 
R  -  Right  click 
K  -  Key  press 
D  -  Double  click 
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Appendix  E:  27  different  ways  of  getting  to  a  file 


Alternative 

Number 

Context 

Choice  within  context 

1 

My  Documents  from  Desktop 

Click  through  folders 

2 

My  Documents  from  Desktop 

Choose  fde  directly 

3 

My  Computer  from  Desktop 

Windows  Explorer 

4 

Open  new  Application  from  Desktop 

Recent  List 

5 

Open  new  Application  from  Desktop 

File  Open  Dialog 

6 

Folder  shortcut  from  Desktop 

Windows  Explorer 

7 

Desktop 

Choose  fde  directly 

8 

My  Computer  from  Start  Menu 

Windows  Explorer 

9 

RUN  from  Start  Menu 

Windows  Explorer 

10 

My  Documents  from  Start  Menu 

Click  through  folders 

11 

My  Documents  from  Start  Menu 

Choose  fde  directly 

12 

Open  application  from  Start  Menu 

Recent  List 

13 

Open  Application  from  Start  Menu 

File  Open  Dialog 

14 

Start  Menu 

Search 

15 

Start  Menu 

My  Recent  Documents 

16 

Address  Bar 

Windows  Explorer 

17 

Address  Bar 

Choose  fde  directly 

18 

Open  folder  from  Quick  Launch 

Windows  Explorer 

19 

Quick  Launch 

Choose  fde  directly 

20 

Open  folder  from  Links  Toolbar 

Windows  Explorer 

21 

Links  Toolbar 

Choose  fde  directly 

22 

Existing  Windows  Explorer 

Choose  fde  directly 

23 

Existing  Windows  Explorer 

Use  Favorites  menu 

24 

Existing  Windows  Explorer 

Search 

25 

Existing  Windows  Explorer 

Click  through  folders 

26 

Existing  Application 

Recent  List 

27 

Existing  Application 

File  Open  Dialog 

Table  1:  27  Ways  of  Getting  To  a  File 
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Appendix  F  -  Field  Visits 

Major:  Major  of  the  subject 

Word-Recent  &  Excel-Recent:  How  many  files  in  their  recent  list  they  visited 
frequently 

Recent  Documents  (Start  Menu):  How  many  files  in  the  recent  documents  did  they 
use  frequently? 


Major 

Word-Recent 

Excel-Recent 

Recent  Docs 

1 .  CS  -  Professor 

1/10 

1/4 

N/A 

2.  Psychology  -  Professor 

2/4 

2/4 

8/15 

3.  Psychology  -  Student 

4/4 

1 

9/15 

4.  CS  -  Office  worker 

1/4 

2/4 

3/15 

5.  CS  -  Office  worker 

0/4 

4/4 

7/15 

6.  Forestry  -  Office  worker 

0/4 

2/4 

2/15 

7.  Business  -  Student 

2/4 

1/4 

5/15 

8.  Forestry  -  Office  worker 

1/4 

1/4 

0/15 

9.  Forestry  -  System  administrator 

0/9 

1/9 

1/15 

10.  Forestry  -  System  administrator 

0/4 

0/4 

0/15 

11.  Forestry  -  Student 

4/4 

2/4 

15/15 

12.  Forestry  -  Student 

3/4 

1/4 

5/15 

13.  Forestry  -  Student 

3/4 

1/4 

9/15 

14.  Biology  -  Student 

2/4 

1/4 

4/15 

15.  Environmental  Sci.  -  Student 

3/4 

0/4 

3/15 

16.  Psychology  -  Professor 

2/4 

1/4 

5/15 

17.  Psychology  -  Student 

1/4 

1/4 

2/15 

18.  Psychology  -  Student 

1/4 

4/4 

5/15 

Table  2:  Files  Used  Frequently 


